Avatar 2 Human Learning: Net Zero and Carbon Negative Emissons

chevron-icon
Back
project-presentation-img
Quincy Johann Sammy
Project Owner

Avatar 2 Human Learning: Net Zero and Carbon Negative Emissons

Funding Awarded

$150,000 USD

Expert Review
Star Filled Image Star Filled Image Star Filled Image Star Filled Image Star Filled Image 0
Community
Star Filled Image Star Filled Image Star Filled Image Star Filled Image Star Filled Image 0 (0)

Status

  • Overall Status

    🛠️ In Progress

  • Funding Transfered

    $112,500 USD

  • Max Funding Amount

    $150,000 USD

Funding Schedule

View Milestones
Milestone Release 1
$5,000 USD Transfer Complete TBD
Milestone Release 2
$10,000 USD Transfer Complete TBD
Milestone Release 3
$10,000 USD Transfer Complete TBD
Milestone Release 4
$10,000 USD Transfer Complete TBD
Milestone Release 5
$10,000 USD Transfer Complete TBD
Milestone Release 6
$10,000 USD Transfer Complete TBD
Milestone Release 7
$10,000 USD Transfer Complete TBD
Milestone Release 8
$11,000 USD Transfer Complete TBD
Milestone Release 9
$12,500 USD Transfer Complete TBD
Milestone Release 10
$12,500 USD Transfer Complete TBD
Milestone Release 11
$11,500 USD Transfer Complete TBD
Milestone Release 12
$15,000 USD Pending TBD
Milestone Release 13
$22,500 USD Pending TBD

Status Reports

Apr. 10, 2023

Status
🙂 Pretty good
Summary

"This milestone is primarily utilitarian as we focus on Speech to Text onboarding for student input, then implementation. We're actually onboarding two services: Google Cloud Services for immediate use, and Whisper open-source onboarding for long term use. We're testing the service with a Kaggle .wav dataset of voice recordings to test transcriptions. The results are looking good. We intend to use this for voice commands such as ""next question"", ""I choose answer A"" and ""please repeat"" to keep the scope manageable. In combination with NLP and perhaps recent remarkable advancements with LLVM, we can see possibilities for later development of free-conversation with avatars trained on the study content. On the avatar side, quality of synthetic voice is important for student comfort and clarity. For text to speech, we've connected the avatars to Azure voices, which provides more than 200 synthetic voices in several nationalities with a quite naturalistic sound. Like other services, these voices could accept SSML tags to speed up, slow down, emulate prosody, and emphasis text. At the mid-point for the metaverse development, it's becoming important to refactor out old models and content. The software suite used for avatar realism and lipsync have also improved quite impressively. Our original models require updating. We're encountering overlapping compatibility issues that impact avatar realism now, so we're taking this opportunity to refactor, before advancing into the ASL module which will be focusing on avatar non-verbal communication. Avatars now speak using Azure voices while delivering tests, presenting content relevant to the site, or other dynamic text. For situations where the text will not change, we can pre-animate lipsync using superior software, then import those keyframes into Unity. When using text to speech, we use a blend of facial animation plus 'live' lip-sync that attempts to follow the waveform shape, and match reasonable lip-sync letter forms in real time. We've added a guide for the new student in the form of a floating round UAV probe. This virtual UAV can also provide aerial views of the site and guide the student to the next avatar with a dialogue to follow. "

Full Report

Jan. 21, 2023

Status
🧐 Fair, but could have been better
Summary

Steady. We would always like to be further ahead; Christmas/New Years and securing ML talent consumed time.

Full Report

Dec. 6, 2022

Status
🙂 Pretty good
Summary

Advancing quickly on the immersive project front. Progress is steady. Machine Learning experts are in high demand, we are a little delayed here. 3D modelling talent we are finding in new graduates, giving them an early opportunity to work and for us, faster model prep than we can do.

Full Report

Oct. 18, 2022

Status
😀 Excellent
Summary

Very well. We are accomplishing more than expected at this early stage, creating worlds and avatars with high quality, and writing many new documents for creating lessons with.

Full Report

Aug. 20, 2022

Status
🙂 Pretty good
Summary

We have completed M-1 and M-0 (milestones). Updated the X2 series reactors in a shipping container build; created instructional avatars (realistic humanoids), created first series of scripts, updated avatar clothes and uniforms, created a downloadable app for laptops, tablets and for VR (Quest 2).

Full Report

Video Updates

Carbix Corporation in collaboration with Xyris Design – Team Meeting 9

31 October 2023

Carbix – Team meeting 8

16 March 2023

Carbix – Team meeting 5

16 March 2023

Carbix – Team meeting 5

16 March 2023

Carbix – Team meeting 2

15 March 2023

Carbix – Team meeting 1

15 March 2023

Carbix – Kickoff presentation

15 March 2023

Project AI Services

No Service Available

Overview

We propose an adaptive AI service API to improve sales and study within virtual worlds by reducing UI and UX friction as natural language processing, behavioral avatar AI, and accessibility intelligence. A partnership between Carbix Corporation and Xyris XR Design provides business acumen, an advanced complex technology, a library of existing meta verse projects, and interactive avatars, giving us a solid foundation for the success of this project. 

Proposal Description

AI services (New or Existing)

Compnay Name

Carbix Corporation in collaboration with Xyris Design. 

Problem Description

Online instruction, by its abstract nature, creates operational barriers to student instruction and sales practices. Transferring lesson plans from paper to web doesn’t communicate context and lacks the hands-on situational experience of on-site instruction, which itself has capacity limitations and additional travel expenses. Accommodations for disability are most often overlooked.  3D virtual worlds and VR simulations use unfamiliar control mechanisms when used by non-gamers, which create distractions while learning complex skills.  An instructor can adapt to the students’ experiences and knowledge to guide instruction and identify areas needing additional learning support.  Personalization, adaptation to learning style and level, accessibility and feedback to the student are areas where we can improve the experience with the toolset provided by Singularity AI in the context of advanced technology operation of the Carbix  X-2, mineral carbonation reactor for carbon capture and utilization.  

Solution Description

We suggest the following development of Singularity AI-based toolsets:

1. Adaptive Learning algorithms to personalize lesson plans to students’ needs.

2. Accessibility Recognition to accommodate disabilities and improve effective communication.

3. Conversational avatars employing text-to-speech in concert with adaptive learning algorithms.

Project Benefits for SNET AI platform

This project will expand the algorithmic options for interaction with diverse students in an immersive environment when training on complex advanced carbon sequestration technology. The addition of an API to Singularity AI will provide a platform for use with Unity and Unreal immersive engines in real time.

 

Competitive Landscape

Strong market fit for global remote distance learning , comprehension and - sales application for carbon capture and utilization for climate mitigation. What we are proposing doesn't exist in the realm of climate mitigation industries, carbon tech or climate tech.  We believe this will provide Carbix and SNET/AI first mover advantage to early users/adopters in this critical industry space.  Our first users/adopters are expected to be from industrial  purchasers (point source emitters) at multinational companies (B2B),  NGO's, academics and government purchasing agencies. 

Marketing & Competition

Our initial marketing plan involves subtle product adverts through display and functionality of the component installed in the Carbix X-2, carbon capture and utilization array. WE can host the application on various platforms, like Roblox. Central to user engagement  is the immersion experience in a real-world setting (background scenes and infrastructure), but through a lens of future-realism.  Each location represents  a prospective opportunity for Carbix’s X2-CCU technology to be deployed. The complex physical hardware and components present an opportunity for the original equipment manufacturers, including Carbix, to showcase their product and how it relates to climate mitigation. The OEM product is essentially a digital twin render that will enable the user to ultimately decide on adopting the technology for CCU, and at the very least, expose the user to hardware and software technology for climate mitigation, they may have not otherwise experienced.  For each component, the OEM pays per predetermined views or pre-determined engagement metrics recorded by algorithms during user-engagement. The ultimate goal for the OEM is to increase their brand awareness in a new metaverse medium and generate sales as awareness evolves to adoption. 

Needed Resources

We would benefit from an experienced individual in Python, R processing, and iterative neural network training methods.

For programming needs, we'd need one or two people experienced with:

  • Software engineering, data formats and algorithmic programming.
  • Data science skills (Python, R, SQL, Java), proficient in math/stats, can develop predictive model strategies.
  • Machine Learning skills (deep learning, neural network architectures, linguistic tagging, video processing, and reinforcement learning.

For the Unity side we’d need:

  • Human animation with emphasis on body language, preferably ASL expert.
  • Data connectivity/Tensorflow/AI Unity expert.

AI Services

Proposal Video

Placeholder for Spotlight Day Pitch-presentations. Video's will be added by the DF team when available.

  • Total Milestones

    13

  • Total Budget

    $150,000 USD

  • Last Updated

    3 Feb 2024

Milestone 1 - Project start

Status
😀 Completed
Description

Signed Contract

Deliverables

Budget

$5,000 USD

Link URL

Milestone 2 - Beta App Release, Phase I

Status
😀 Completed
Description

1. Scene updates for facility and detailed Lesson scripts for current equipment. Additional avatar dialogue scripts for X2 and CCU provided in text or spreadsheet form. 2. VR Headset models reference list. Immersive scene model updates. 2X new avatars (female). Corrections to avatar posture. 3. Desktop .exe recompilation of Geofloat scene. 4. Additional PCVR (tethered Oculus Quest) VR-Desktop .exe compilation.

Deliverables

Budget

$10,000 USD

Link URL

Milestone 3 - Beta App Release, Phase II

Status
😀 Completed
Description

1. Nature based scenes of cement plants and X2-CCU installation. Training scripts/syllabi for facility/layout for avatars. Pop up guidance for users. Compiled Windows .exe. 2. New project scene with Iceland Geothermal and embedded training scripts. Compiled Windows .exe.

Deliverables

Budget

$10,000 USD

Link URL

Milestone 4 - Module 1 (Adaptive Learning) Phase I

Status
😀 Completed
Description

1. Lean Strategy Doc (Adaptive Learning, Learning graph ontology examples. Web-facing lesson creator UI proposals) 2. Two Dynamic models for CCU: Reactor and ID (induced draft) fan system. PC and PCVR exes of new CCU models.

Deliverables

Budget

$10,000 USD

Link URL

Milestone 5 - Module 1 (Phase II)

Status
😀 Completed
Description

SNET API delivering lesson graphs and Recommendations (rev 1 MVP). Please refer to attachment “Anticipated API Overview – Module 1 Adaptive Learning”. VR grab/manipulate object functionality.

Deliverables

Budget

$10,000 USD

Link URL

Milestone 6 - Module 1 (Phase III)

Status
😀 Completed
Description

1. Add X2- CCU for geo-thermal with nature and technical build scenes. 2. New .exe of Cement and geothermal scenes utilizing Carbix models. 3. Improvement of Adaptive Learning recommendation weighting algorithms. 4. (SNET API Adaptive Learning Rev 2 MVP). Please refer to attachment “Anticipated API Overview – Module 1 Adaptive Learning”.

Deliverables

Budget

$10,000 USD

Link URL

Milestone 7 - Module 2 (Conversational Avatars) u2013 Phase I

Status
😀 Completed
Description

Lean Strategy Doc (Integration of open-source speech recognition library and linguistic tagging library). Speech to text SNET api rev 1 MVP. Please refer to attachment “Anticipated API Overview – Module 2 Speech Recognition”.

Deliverables

Budget

$10,000 USD

Link URL

Milestone 8 - Module 2 (Phase II)

Status
😀 Completed
Description

Command recognition (fx “Please repeat”, “Tell me more”, “I got it”). Avatar text responses returned via API which further the lesson guidance and confirm user input. Conversational Avatar SNET api rev 2 MVP. Please refer to attachment “Anticipated API Overview – Module 2 Speech Recognition”.

Deliverables

Budget

$11,000 USD

Link URL

Milestone 9 - Module 3 (ASL Recognition) Phase I

Status
😀 Completed
Description

Document: project strategy plan to create ASL recognition focusing on interpreting lesson plan phrases, confirmation commands, static letters and dynamic phrases. ML solution description and desired outcomes.

Deliverables

Budget

$12,500 USD

Link URL

Milestone 10 - Module 3 (Phase II)

Status
😀 Completed
Description

Motion capture library of alphabet and lesson content with recorded video of human ASL translator (including facial expressions). Initial machine learning implementation. A test showing responsiveness of system to webcam ASL input under ideal lighting conditions, finger markers and controlled speed of human translator’s arm motion.

Deliverables

Budget

$12,500 USD

Link URL

Milestone 11 - Module 3 (Phase III)

Status
😀 Completed
Description

ASL to Text SNET API rev 1 MVP. Please refer to attachment “Anticipated API Overview – Module 3 ASL Recognition”. Functionality: ASL-to-text, learner confirmation commands, pipeline from ASL-to-text results to linguistic tagging, piping of results through Adaptive Learning graph, Adaptive learning graph returning Lesson recommendation and Avatar conversation text (Round Trip of lesson delivery / student choice / ASL recognition / return via SNET API of lesson recommendation).

Deliverables

Budget

$11,500 USD

Link URL

Milestone 12 - Marketing I

Status
😐 Not Started
Description

Post dev – Marketing on SNET, primarily for API calls.

Deliverables

Budget

$15,000 USD

Link URL

Milestone 13 - Marketing II

Status
😐 Not Started
Description

Post dev – Marketing on SNET, primarily for API calls and user engagement/focus groups metrics.

Deliverables

Budget

$22,500 USD

Link URL

Join the Discussion (0)

Reviews & Rating

New reviews and ratings are disabled for Awarded Projects

Sort by

0 ratings

Summary

Overall Community

0

from 0 reviews
  • 5
    0
  • 4
    0
  • 3
    0
  • 2
    0
  • 1
    0

Feasibility

0

from 0 reviews

Viability

0

from 0 reviews

Desirabilty

0

from 0 reviews

Usefulness

0

from 0 reviews

Get Involved

Contribute your talents by joining your dream team and project. Visit the job board at Freelance DAO for opportunites today!

View Job Board