The 9th edition of TADHack Global was conducted on October 15-16, 2022. Over 450 hackers from around the world participated, creating a total of 52 hacks for submission. 

My name is Fahad Mahmood and I am new to the WebRTC development team at WebRTC.ventures. Along with two of my friends, I participated in this year’s hackathon. Our project received the “Best Disruptive Digital Engagement Hack” prize!

Track Selection

We already had an idea in advance of the hackathon. So for us, choosing a track was finding the CPaaS platform which best suited our needs. As we were participating remotely, we had three global sponsor tracks that we could choose from. The local sponsors were for in-person participants.

Jambonz is an open source CPaaS platform. They partnered with Cognigy (a low-code voice gateway platform) for the hackathon and provided early access to their integration. 

For participation in the Strolid track, hackers had to submit audio jokes for vCon (virtual conversation). Just like iCal is for sharing appointments and XLS for worksheets, vCon is a new open standard for sharing conversation data like a transcript, audio, video, timestamps metadata, etc.

Finally, the Radisys Engage track provided access to their engagedigital.ai platform for participation in the hackathon. There were tons of programmable voice, video, and analytics tools that hackers could choose from and use in their hacks. Some of them were Intelligent IVR, Flexible (P2A / A2P / Multiparty) Calling, Speech Analytics, Video Assistant, Video Collaboration, etc.

We chose the Radisys Engage sponsor for our hack. Among other tools, their Multi-Party Video Calling MCU and Video Assistant were best suited for our needs and we could integrate them into our hack easily. 

Our Idea

I think it’s time to answer the most intriguing question: What was your idea for the hackathon, Fahad?

I have been following the blockchain ecosystem for more than 1.5 years. Not only have I seen quite a few metaverse-related projects, but I have some experience writing smart contracts and developing decentralized applications myself.

So, we decided to build a metaverse and real-time engagement platform called LifeVerse that would connect users with healthcare professionals. People can visit different clinics, interact and socialize with the staff, book appointments and consult with healthcare professionals through a single metaverse platform. This idea could be expanded for other use cases, like EdTech solutions.

We had very limited time to work on the hack. So, we decided to initially build only one clinic in the metaverse platform where a patient could schedule an appointment using Radisys Engage Video Assistant and then consult a healthcare professional using their Multiparty Conference MCU. 

Technologies and Architecture

The first step was choosing the right technologies for the implementation. Since all of us were experienced in JavaScript, we decided to use ReactJS for the frontend development and NodeJS for the backend server. 

We used React Three Fiber (React renderer for three.js) to build our 3D metaverse. Adobe Mixamo and Blender for 3D Models and characters. 

The backend server receives requests from the clients through the REST API, authenticates users, and then communicates with the Radisys Engage MCU through their REST API to create different rooms and generate temporary access tokens for the clients. Then, the client can use the Radisys Engage SDK to join the room using the token received from the backend server.

Implementation

Our plan was quite simple. 

We focused on setting up our infrastructure on Day One. We spent most of our time learning different 3D tools and frameworks like Blender, Adobe Maximo, and three.js. By the end of this day, we were able to create our mini metaverse world with one health clinic and one 3D character which could perform different animations like walking, running, dancing, etc. 

On Day Two, we first experimented with the Radisys Digital Engage platform and looked into their documents and Rest API for different services. To implement the scenario where a person consults with a healthcare professional through the metaverse, we first created a web server that communicates with the MCU using REST API to manage multiparty conferences and generate access tokens for the clients using NodeJS and ExpressJS.

We connected our React frontend with the web server to request access tokens for the MCU, then we used Radisys Engage SDK to establish connections, join rooms, share local streams, and receive incoming streams from the MCU server. We didn’t have time to implement the Video Assistant so we spent our remaining time creating the pitch video and submitting it before the deadline.

Pitch and Demo

Here is the pitch video of LifeVerse that we submitted for the hackathon:

LifeVerse by Fahad Mahmood , Sarmad Faheem, and Faheem Azhar

Hooray! We won!

After two tiring but fun days, we were finally able to submit our project for the hackathon. Overall it was an amazing experience — we learned so much in just two days! Then it was time to wait for the result.

Hooray! We won a special category “Best Disruptive Digital Hack” prize. We were among six global winners from 23 hacks submitted for the Radisys Engage track.