Audio Listening Device to Improve Surgical Outcomes
Remote monitoring of surgeries is another technological advancement in the world of medicine that provides a multitude of benefits. It protects the patient, surgeon, and healthcare facility by monitoring compliance with defined best practices.
Samson Nkrumah, WebRTC.ventures Developer
To achieve near-real-time audio processing, we leveraged the Amazon Chime SDK to manage both segmented audio files and seamlessly concatenate them into a unified stream as soon as the session concludes. The most intriguing aspect was the implementation of speech diarization and speaker differentiation post-session. To accomplish this, we integrated a suite of AWS services, including EventBridge, SQS, and the Amazon Chime SDK, in conjunction with Symbl.ai. This intricate orchestration enabled a streamlined flow of data in order to successfully achieve the speech diarization.
The Needs
The client, a healthcare startup focused on digital innovation, wanted to develop a listening device to track dialog in an operating room that would be transcribed in real time, with a postoperative transcript and an aggregated dashboard for oversight.
We designed a front-end web application, REST API backend, and database to track and surface key moments in surgical procedures, partnering with Symbl.ai to provide real-time transcription and conversational insights.
Audio communication
Auto reconnect
Cache storage
Enable recordings
Integration with Machine Learning/AI services
Pre-call connectivity checks
Scalable WebRTC infrastructure
Store recordings in the cloud
Text transcription
Tech Stack
- Amazon Chime SDK for real-time audio recording
- Symbl.ai for speech to text, sentiment analysis capture and display, and summary
- AWS Lambda for serverless infrastructure
- AWS SQS for message queuing
- Amazon CloudWatch for monitoring
- Amazon EventBridge for logging
- AWS CloudFront for static content distribution
- AWS Elastic Container Service (ECS) for scalable dockerized backend
- AWS S3 for files, recording and media storage
- Docker for containerizing
- Github for CI/CD
- GoDaddy for the TLS certs and application domains
- MongoDB for the database thanks to its flexible schema, well-suited for applications where the data structure can evolve over time, and real time data capabilities. It can handle large volumes of data and provide near real-time querying.
- NodeJS to use Javascript at client and server, making it easier sharing code and expertise between frontend and backend development
- Terraform for Infrastructure as Code
- TypeScript/React thanks to its better tooling and IDE support, enhanced developer experience and growing ecosystem
Let's turn your project into another story of success.
Client:
Under NDA
Type of Application:
Audio Listening Device to Improve Surgical Outcomes
How We Helped:
We built a branded, cutting edge WebRTC audio application leveraging the Amazon Chime SDK designed to track and surface key moments in surgical procedures, partnering with Symbl.ai to provide real-time transcription and conversational insights.
Features:
- Audio capture and controls
- Real-time transcription and storage
- Post-operative transcription and storage
- Asynchronous speaker separation
- Real-time sentiment analysis to understand emotional context
- Custom trackers to detect and classify keywords, phrases, intents, and more
- Generates conversational summaries and extracts questions
- Surgeon rating and audio recording rating
- Aggregated dashboard for oversight
- Language support for EN US/UK (real-time) and Arabic
- Ongoing application support through Managed Service contract
Let’s convert your app into another story of success!