Think Inclusion Conferencing App

How Our Project Will Contribute To The Growth Of The Decentralized AI Platform

One of the goals of the BGI Nexus is to develop a platform that enables anyone, anywhere, to determine what a beneficial and inclusive future for humankind and the earth should be, this panders directly with the Think Incusion Conferencing app, because it sorts to bring a marginalised group of persons, in this case those with audio and speech impairment to the conventional digital space, creating a future of mutual shared digital space for them.

Our Team

Ubio Obu: Team Lead: CEO of Remostart and an AI Engineer with several publications in several Journals like IEEE, American Institute of Physics etc

Ediyangha Ottoho: Full Stack Developer: More than 10 years of development experince and has worked on several top projects where some have morphed into Unicorns.

Daniel Effiom: Project and Ethics Manager: He is the Chief operating officer at Remostart and has high value for ensuring projects are executed on time and following a high level of standards.

Company Name (if applicable)

The core problem we are aiming to solve

The core problem we aim to solve is the digital exclusion of over 450 million people with speech impairments, as highlighted by WHO. While physical spaces have become more inclusive, digital platforms remain largely inaccessible, limiting participation in virtual interactions. Existing assistive tools are often prohibitively expensive, creating a significant barrier for millions who wish to be included and have their voices heard. This lack of affordable, inclusive technology deprives them of equal opportunities in communication, collaboration, and decision-making.

Our specific solution to this problem

Our solution is an AI-powered meeting app that bridges the communication gap for speech-impaired individuals by integrating a real-time sign language translator. The system converts spoken conversations into sign language, allowing the speech-impaired user to follow discussions effortlessly. In return, the user's sign language responses are translated into synthesized speech, enabling seamless participation as if they were speaking audibly.

Key Features include:

• Interruptions & Turn-Taking: A gesture-based or on-screen prompt system ensures speech-impaired users can indicate when they want to contribute.
• Voice Representation: Users can select or customize an AI-generated voice that aligns with their identity.
• Multi-Mode Accessibility: The app supports text-based inputs for those preferring typing and integrates haptic feedback for additional engagement.
• Affordability & Scalability: Leveraging AI and open-source technologies keeps the solution cost-effective, ensuring wider adoption.

Project details

Solution Overview

The Think Inclusion Meeting App is an innovative AI-powered communication platform designed to bridge the accessibility gap for individuals with speech impairments. It ensures full participation in virtual meetings by integrating a real-time sign language translator that converts spoken conversations into sign language and, in return, translates sign language responses into natural-sounding synthesized speech. This enables speech-impaired individuals to engage in discussions as seamlessly as their hearing and speaking counterparts.

The system leverages computer vision, NLP, and speech synthesis to interpret gestures, process spoken language, and generate a smooth and natural audio output. It ensures that users can interrupt, respond, and contribute in meetings without being sidelined. Additionally, the tool provides multiple accessibility modes, including text-based input, haptic feedback, and voice customization, allowing users to personalize their communication experience.

Technical Architecture

The solution consists of multiple integrated components that work together to ensure a fluid, real-time experience:

1. Input Layer

• Spoken Conversations: Captured via the app’s audio input, transcribed using an automatic speech recognition (ASR) model like Whisper or DeepSpeech.
• Sign Language Interpretation: The system uses real-time video analysis powered by computer vision models like OpenPose or MediaPipe to track hand movements and facial expressions. These inputs are processed using a sign language recognition model trained on datasets like ASLLVD (American Sign Language Video Dataset) and PHOENIX-2014T.
• Text Input: Users can type responses if they prefer a non-gesture-based approach.

2. Processing Layer

• Natural Language Processing (NLP): Used to structure and interpret transcribed speech and sign language responses.
• Gesture-to-Text Translation: The AI model maps hand and facial movements to a structured text representation of sign language.
• Response Handling: Users can signal interruptions using predefined gestures or on-screen prompts, ensuring active participation.
• Voice Personalization: A text-to-speech (TTS) module, using models like Tacotron 2 or VITS, converts structured sign-language-translated text into a natural voice output.

3. Output Layer

• Sign Language Visualization: The system renders a sign language avatar for hearing users who prefer to receive signed translations.
• AI-Generated Speech Output: The user’s translated text is synthesized into speech and broadcasted in the meeting.
• Haptic Feedback: Tactile notifications assist speech-impaired users in staying engaged.

4. Cloud & Edge AI Infrastructure

• Cloud-Based AI Models: Speech-to-text, sign recognition, and text-to-speech are deployed on cloud inference servers for efficient processing.
• On-Device Computation: Edge AI models optimize low-latency processing on mobile devices for responsive real-time interactions.

How It Works

1. A spoken conversation begins. The app's ASR model transcribes speech in real time.
2. Sign language translation initiates. The AI model converts the transcript into animated sign language, displayed on-screen for the speech-impaired participant.
3. Speech-impaired users respond using sign language. The AI recognizes and translates these signs into structured text.
4. Text is converted into speech. The system synthesizes the response into natural-sounding audio, allowing the speech-impaired individual to contribute seamlessly.
5. Users can signal to speak. A gesture-based system or on-screen button allows them to indicate when they wish to participate.
6. A fully inclusive meeting unfolds. The AI ensures all users, regardless of impairment, communicate in a natural and synchronized manner.

Uniqueness of the Solution

• Bidirectional Accessibility: Most tools cater to one-way accessibility (e.g., speech-to-text), whereas this app ensures two-way communication by translating speech to sign language and sign language to speech.
• Customizable AI Voice Representation: Unlike traditional systems, users can select a preferred AI-generated voice for natural, personalized communication.
• Interrupt Mechanism for Equal Participation: A robust gesture-based interruption system ensures speech-impaired individuals can engage in discussions in real time.
• Affordable & Scalable: Many existing assistive tools are prohibitively expensive. By leveraging AI-driven, open-source components, this tool remains cost-effective and widely accessible.

Social Impact

1. Workplace Inclusion: Enables professionals with speech impairments to actively participate in meetings, fostering diverse and inclusive workplaces.
2. Education & Learning: Improves accessibility in online classrooms, ensuring equitable learning opportunities.
3. Telemedicine & Healthcare: Allows patients with speech impairments to communicate effectively with doctors in virtual consultations.
4. Public & Civic Engagement: Promotes inclusivity in government meetings, public discussions, and community events, ensuring no voice goes unheard.
5. Remote Work & Digital Communication: Bridges the gap for speech-impaired individuals in remote work settings, allowing them to engage in virtual teams without barriers.

Ethical Considerations

• Bias in AI Training: Ensuring sign language models are trained on diverse datasets covering multiple sign languages (ASL, BSL, etc.) to prevent biases and misinterpretations.
• User Data Privacy: Implementing strict data encryption and edge AI processing to minimize storage and sharing of sensitive communication data.
• Cultural Sensitivity in Sign Language: Sign languages differ across regions, so the system must adapt dynamically based on the user's geographic and linguistic context.
• Consent-Based AI Interactions: Users must have full control over their data, AI-generated voice selection, and system preferences.
• Non-Disruptive Integration: The system should function as an assistive layer within existing meeting platforms (Zoom, Google Meet, Microsoft Teams), ensuring easy adoption

Existing resources

We have a repository of more than 5000 tech talents in our platform, who will act as beta testers for our solution, helping us improve the solution.

Open Source Licensing