April 2, 2025
Comments Off on Building an Interactive Emoji Expression Game with LiveKit Video Conferencing

Video conferencing can be more than just communication—it can be an interactive gaming experience! We built a web application that connects players via LiveKit open source video conferencing and challenges them to an emoji-matching game using the FaceAPI for real-time facial expression analysis. We call it “FaceOff”.

In this first post of the series, we’ll cover the basics of using LiveKit and how we leveraged it to build this web-based video conference game. Our focus will be on the technical implementation: setting up peer-to-peer video communication, managing connection tokens, and establishing a robust mechanism for sending game-related data between participants.

In a later post, we will go deeper into the face recognition and expression matching the emoji and the actual game logic for this application.

Requirements and Stack

You will need the following to complete this tutorial:

  • A LiveKit Cloud account (you can use the free Build plan)
  • NPM
  • Node.js
  • Git
  • A terminal emulator application
  • Your favorite code editor

To communicate with LiveKit Cloud we will need:

  • API Key
  • API Secret
  • LiveKitHost

The host will be available after you create your LiveKit Cloud account, but you will need to manually create your API Key and API Secret using the LiveKit CLI or manually in the interface.

With the environment variables copied, you’ll need to create a file in the root of the application named .env and paste the variables there.

The application code is written in:

  • ReactJS for the frontend
  • Node for the backend
  • Dotenv to managing environment variables

You can download the sample repository locally using git. Open your favorite terminal, navigate to where you want to download the code and run the following command:

git clone https://github.com/WebRTCventures/faceoff && cd faceoff

Using LiveKit

You can integrate LiveKit into your application in two different ways. Hosting your own LiveKit server provides for maximum control and customization. Alternatively, the LiveKit Cloud is a fully-managed, globally distributed service with automatic scaling and high reliability. To make this post easier to follow, we will go with the second option

LiveKit provides helper libraries for both frontend and backend code. In the frontend, LiveKit makes available a list of UI components that we use to build our interface. For the backend, we use the LiveKit SDK to manage authentication with the LiveKit platform, and also to manage game state for all the players. The SDK has good documentation to make things easier for us.

Let’s start by setting things up in our backend!

LiveKit Backend

Token handling

The first step is to enable users to join the game room, to do so they need a unique token. The idea is that every time a peer tries to enter a room, it asks our backend server for that credential.

To do so, we need an endpoint where the frontend can request token information by passing a username. The response will include a token and a roomId. The roomId is returned so that the first peer can share it with others, enabling them to connect to the same room. When releasing to production, you’d want to put such an endpoint behind a proper authentication & authorization mechanism, but that’s beyond the scope of this post so we won’t cover it. 

The roomId can also be a parameter in the request. If the value is null, there is no room created and we need to create it. If we receive an id, then we simply connect that peer to the existing room:

// server.js

app.get('/getToken', async (req, res) => {
  const userName = req.query.userName;
  const room = req.query.roomId;

  try {
    const { token, roomId } = await handleToken(userName, room)
     res.json({token, roomId});
 } catch(e) {
     console.error(e);
     res.status(500).json({error: e});
 }
});

The handleToken function does the logic to check if there is a roomId. If it is not received, it creates a new roomId using uuid.

// server.js

const handleToken = async (participantName, roomName) => {
   // if the room is already created, we will receive  name
   // if not we will create a new roomId using uuid

   const room = roomName || uuidv4();
   ...
   return await createToken(participantName, room);
};

Then, we create the token using the participant name and the room. LiveKit SDK has a class called AccessToken which receives the API key, secret and an object of options. In those options we can send the identity of each peer so we can keep track of them later.

After the creation of the access token, we need to add it to a room and give grants to it using the function addGrant.

Once the grant is added, you will need to return it as a JWT so the frontend can use that token to communicate with the React Components offered by LiveKit.

// server.js
const createToken = async (participantName, room) => {
  const at = new AccessToken(
    process.env.LIVEKIT_API_KEY,
    process.env.LIVEKIT_API_SECRET,
    {
      identity: participantName,
      ...
    }
  );
  at.addGrant({ roomJoin: true, room });
  return { token: await at.toJwt(), roomId: room };
}

That’s it to create a token with LiveKit! And this is all you need to start using the token with your components in the front end.

Randomizing emojis

Emojis are crucial in our game, therefore, we also need an endpoint to ask for the current emoji. We do this on the backend, so when one user clicks to start the game, we send a request to this endpoint, which in turn sends a message with the chosen emoji through the data channel to all users.

In the backend, we send data through a data channel using the sendData method from RoomServiceClient. All we need to do is to configure a message topic. 

The endpoint receives usedEmojis from the frontend, which tracks emojis from previous rounds to prevent repetition. It also receives roomId to identify the target room for the message.

// server.js
app.get('/getEmoji', async (req, res) => {
  const room = req.query.roomId;
  const usedEmojis = req.query.usedEmojis;
  const roomService = new RoomServiceClient(
    process.env.LIVEKIT_URL,
    process.env.LIVEKIT_API_KEY,
    process.env.LIVEKIT_API_SECRET
  );
  const encoder = new TextEncoder()
  const emojis = [
    //   😀           😐            🙁           😮           
    '\u{1F600}', '\u{1F610}',  '\u{1F641}', '\u{1F62E}', 
    //   😰           😡           🤢
    '\u{1F630}', '\u{1F621}', '\u{1F922}'
  ]
  // removes the used emojis for randomizing
  const unusedEmojis = emojis.filter(
    (emoji) => !usedEmojis.includes(emoji)
  );
  let emoji = unusedEmojis[
    Math.floor(Math.random() * unusedEmojis.length)
  ];
  const strData = JSON.stringify({emoji})
  const data = encoder.encode(strData);
  // sends the chosen emoji via data channel to the room
  await roomService.sendData(
    room, data, null, {topic: 'chat'}
  );
  res.status(200).json({});
})

To allow our frontend to communicate with both LiveKit and our backend, we set up the REACT_APP_LIVEKIT_URL and REACT_APP_SERVER_URL environmental variables in the .env file. Note the REACT_APP_ prefix for both of these, this is required for such environments to be available for the React application.

Make sure you add your own values for these variables. If you are running the backend locally you can add the value “http://localhost:3001” for REACT_APP_SERVER_URL.

Game Frontend

Live Kit offers you a lot of components to create your video conference, as well as hooks to help you with your integration.

Let’s take a look at how we can use these to build the UI of our game.

PreJoining the Game

We are using the PreJoin component as the home of our app. This allows you to ask permission for the video, audio and name of the user. This component has a callback for the submission of the form that returns the set of chosen options for the user. We use such options to fetch the token from our backend.

// src/components/Home.js

export default function Home() {
 ...

 function handleSubmit(userOptions) {
   setUserOptions(userOptions);
   fetchToken(userOptions.username);
 }

 ...
 return (
  ...
  <PreJoin
    joinLabel="Join a Game"
    userLabel="Enter your name to continue"
    onSubmit={handleSubmit}
  />
 )
}

The PreJoin does not run inside a room, so we won’t need a roomID for this part. However, to render the game room that will contain participants’ video and audio, we will need to wrap it with a LiveKitRoom component. This component will give you access to a bunch of hooks and components to render and handle peers’ information. This is where we pass the token and LiveKit server URL.

// src/components/Home.js


export default function Home() {
 ...
 return (
  { token ? (
    <LiveKitRoom
      audio={userOptions.audioEnabled}
      video={userOptions.videoEnabled}
      token={token}
      serverUrl={process.env.REACT_APP_LIVEKIT_URL}
    >
     <GameRoom roomId={roomId} userOptions={userOptions} />
    </LiveKitRoom>
   ) : (     
    <PreJoin
      joinLabel="Join a Game"
      userLabel="Enter your name to continue"
      onSubmit={handleSubmit}
    />
   ) }
 )
}

The Game Room

The GameRoom component handles the game stat: whether the game has started or waiting to start the round. We control this state using the React’s state functionality. If the game state is waiting, we just render our VideoCall component so the peers can talk with each other. If the game is playing, we show our EmojiRoom component to run the game logic.

When we click to start the game, we need to let the other peers know that the game has started. We do this by sending a message in the data channels through the media server. LiveKit offers a really easy way to do this using the useDataChannel hook. All we need to do is “subscribe” to a specific topic, send messages through it, and decode the messages on the receiver side to check their content.

// src/components/GameRoom.js
export default function GameRoom({roomId, userOptions}) {
  const [gameState, setGameState] = useState('waiting');
  // subscribe to the "chat" topic and decode the messages
  const { message: latestMessage, send } = useDataChannel(
    "chat", 
    (msg) => {
      const decodedMsg = new TextDecoder("utf-8").decode(
        msg.payload
      )
      if(decodedMsg === 'game started') {
        setGameState('playing');
      } 
      ...
    }
  );
  
  function startGame() {
    ...
    // set the state for the current client
    setGameState('playing');
    // send the message to the other peers 
    send(new TextEncoder().encode('game started'))
  }
  
  return (
    ...
    { gameState === 'playing' && 
      <EmojiRoom 
        username={userOptions.username} 
        emoji={currentEmoji} 
        endGameFn={endGame} 
      /> 
    }
    { gameState === 'waiting' && 
      <VideoCall 
        roomId={roomId} 
        participants={participants} 
      /> 
    }
    ...
  )
}

In the VideoCall we use two components from LiveKit RoomAudioRenderer, which handles all the room-wide audio for you, and ControlBar, which renders the default controls for you video call (mute/unmute, camera on/off, share screen and leave).

The VideoConference component is where we actually render our peers. We will check that next.

// src/components/VideoCall.js

export default function VideoCall({roomId}) {
  return (
    ...
    <VideoConference roomId={roomId} />
    <RoomAudioRenderer />
    <ControlBar variation='verbose' />
  )
}

The useTracks is a hook that returns all camera and screen share tracks for all peers. Once we get these, we send them to a GridLayout, which displays the children in a grid layout. The child is used as a template to render all passed in tracks. In this case we are just sending the ParticipantTile as the child, to render all cameras and screen shares of the peers in the room.

// src/components/VideoConference.js

export default function VideoConference() {
 const tracks = useTracks(
   [
     { source: Track.Source.Camera, withPlaceholder: true },
     { source: Track.Source.ScreenShare, withPlaceholder: false },
   ],
   { onlySubscribed: false },
 );
 return (
   <GridLayout tracks={tracks}>
     <ParticipantTile />
   </GridLayout>
 );
}

Running the application

Now that we have our backend and frontend ready, let’s run our application to check how it is going!

First, you will need to install all the dependencies for the app:

npm install

You will need to run both separately in order to work. So, for the backend run this command in the root directory:

node server.js

To run the frontend application, run this in the root directory as well:

npm install && npm start

And that’s the video application piece!

This is all you need to know to create your own video application with LiveKit. Of course, our game is not ready to play yet. We still need to add the face recognition in order to score each expression from the user, but that is a talk for our next post. Stay tuned!

WebRTC.ventures is proud to be a LiveKit Development Partner. We can help you build out a new application with LiveKit, or convert an existing solution over to LiveKit. Contact WebRTC.ventures today!

Recent Blog Posts