This guide provides you with an overview of how to build your video application with the Twilio Programmable Video iOS SDK.
If you haven't already done so, take a look at the open source video collaboration app and quickstart apps. Then come back to this guide for more detail on how to add video to your own app.
Let's start with an overview of the Programmable Video API:
Room
represents a real-time audio, data, video, and/or screen-share session, and is the basic building block for a Programmable Video application.Participants
represent client applications that are connected to a Room and sharing audio, data, and/or video media with one another.Tracks
represent the individual audio, data, and video media streams that are shared within a Room.LocalTracks
represent the audio, data, and video captured from the local client's media sources (for example, microphone and camera).RemoteTracks
represent the audio, data, and video tracks from other participants connected to the Room.The following code samples illustrate common tasks that you as a developer may wish to perform related to a Room and its Participants.
To start using the iOS Programmable Video SDK in your apps, you need to perform a few tasks first.
The Twilio Video iOS SDK dynamic framework can be installed using Swift Package Manager, CocoaPods or manually, as you prefer.
You can add Programmable Video for iOS by adding the https://github.com/twilio/twilio-video-ios
repository as a Swift Package.
In your Build Settings, you will also need to modify Other Linker Flags
to include -ObjC
.
As of the latest release of Xcode, there is a known issue with consuming binary frameworks distributed via Swift Package Manager. The current workaround to this issue is to add a Run Script Phase
to the Build Phases
of your Xcode project. This Run Script Phase
should come after the Embed Frameworks
build phase. This new Run Script Phase
should contain the following code:
1find "${CODESIGNING_FOLDER_PATH}" -name '*.framework' -print0 | while read -d $'\0' framework2do3codesign --force --deep --sign "${EXPANDED_CODE_SIGN_IDENTITY}" --preserve-metadata=identifier,entitlements --timestamp=none "${framework}"4done5
1source 'https://github.com/CocoaPods/Specs'23platform :ios, '12.2'45target 'TARGET_NAME' do6pod 'TwilioVideo', '~> 5'7end
Then run pod install
to install the dependencies to your project.
TwilioVideo.xcframework
is distributed as a dynamic iOS framework that you can drag and drop into your existing projects.
View all Video iOS Releases here or just download the latest Video dynamic framework here.
Once you've downloaded and unpacked the XCFramework, navigate to your Xcode project's General settings page. Drag and drop TwilioVideo.xcframework
onto the Frameworks, Libraries, and Embedded Content section. Ensure that "Copy items if needed" is checked and press Finish. Ensure that "Embed & Sign" is selected.
In your Build Settings, you will also need to modify Other Linker Flags
to include -ObjC
.
To allow a connection to a Room to be persisted while an application is running in the background, you must select the Audio, AirPlay, and Picture in Picture
background mode from the Capabilities
project settings page.
Twilio Video requires user permission for features like sharing video from the camera or audio from the microphone. Consider how your applications might function with reduced functionality in case some permissions (like the camera) are declined.
Friendly Name | Privacy Usage Description Key | iOS Version | Recommendation | |
---|---|---|---|---|
Camera | NSCameraUsageDescription | All | Request permissions using AVCaptureDevice.requestAccess(for:completionHandler:) before calling TwilioVideoSDK.connect(options:delegate:) with a TVICameraSource. | |
Microphone | NSMicrophoneUsageDescription | All | Request permissions using AVAudioSession.requestRecordPermission() before calling TwilioVideoSDK.connect(options:delegate:) when Participants publish or subscribe to audio tracks with TVIDefaultAudioDevice. (*) | |
Local Network | NSLocalNetworkUsageDescription | 14.0 + | This permission is not recommended unless your app is already using NWConnection APIs on the local network. |
The iOS SDK supports iOS 12.2 or higher. It is built for arm64 and x86_64 architectures with Bitcode slices for arm64 devices.
The TwilioVideo.framework
is built with Xcode 13. The framework can be consumed with previous versions of Xcode. However, re-compiling Bitcode when exporting for Ad Hoc or Enterprise distribution requires the use of Xcode 13.x.
API Keys represent credentials to access the Twilio API. You use them to:
Follow the instructions in the API Keys Overview doc to create a new API key for your project.
To execute the code samples below, you'll need to generate an Access Token. An Access Token is a short-lived credential used to authenticate your client-side application to Twilio.
You can generate an Access Token using either the Twilio CLI or a Twilio Helper Library.
For application testing purposes, the Twilio CLI provides a quick way to generate Access Tokens that you can then copy/paste into your application. In a production application, you should use the Twilio Helper Libraries because your back-end server will need to generate an Access Token for every user in your application.
To use the CLI, you will need to install the Twilio CLI and log in to your Twilio account from the command line; see the CLI Quickstart for instructions. Then, you can install the Token CLI plugin with the following command:
twilio plugins:install @twilio-labs/plugin-token
To generate an Access Token, run the following command. --identity
is a required argument and should be a string that represents the user identity for this Access Token.
twilio token:video --identity=<identity>
You can find examples of how to generate an Access Token for a participant using Twilio's Helper Libraries in the User Identity and Access Token guide.
Call TwilioVideo.connect()
to connect to a Room from your iOS application. Once connected, you can send and receive audio and video streams with other Participants who are connected to the Room.
1@IBAction func createARoom(sender: AnyObject) {2let connectOptions = ConnectOptions(token: accessToken) { (builder) in3builder.roomName = "my-room"4}5room = TwilioVideoSDK.connect(options: connectOptions, delegate: self)6}78// MARK: RoomDelegate910func roomDidConnect(room: Room) {11print("Did connect to Room")1213if let localParticipant = room.localParticipant {14print("Local identity \(localParticipant.identity)")1516// Set the delegate of the local particiant to receive callbacks17localParticipant.delegate = self18}19}
You must pass the Access Token when connecting to a Room. You may also optionally pass the following:
The name of the Room specifies which Room you wish to join. If you have enabled client-side Room creation for your Account and a Room by that name doesn't already exist, it will be created upon connection. If a Room by that name is already active, you'll be connected to the Room and receive notifications from any other Participants also connected to the same Room. Room names must be unique within an Account.
If you have enabled client-side Room creation, or ad hoc Room creation, any new Room you create via the iOS SDK will follow the default Room settings that you've specified in your Account. These settings include options like a StatusCallback
URL where you can receive Room creation and other webhook events, the maximum number of Participants, automatic recording, and more. You can view and update your default Room settings in the Twilio Console.
You can also create a Room using the Rooms REST API. Look at the REST API Rooms resource docs for more details.
If you'd like to join a Room you know already exists, you handle that exactly the same way as creating a room: just pass the Room name to the connect
method.
Once in a Room, you'll receive a room:participantDidConnect:
callback for each Participant that successfully joins. Querying the remoteParticipants
getter will return any existing Participants who have already joined the Room.
1@IBAction func joinRoom(sender: AnyObject) {2let connectOptions = ConnectOptions(token: accessToken) { (builder) in3builder.roomName = "existing-room"4})5room = TwilioVideoSDK.connect(options: connectOptions, delegate: self)6}78// MARK: RoomDelegate910func roomDidConnect(room: Room) {11print("Did connect to room")1213if let localParticipant = room.localParticipant {14print("Local identity \(localParticipant.identity)")1516// Set the delegate of the local particiant to receive callbacks17localParticipant.delegate = self18}19}
You can capture local media from your device's microphone, camera or screen-share on different platforms in the following ways:
In an iOS application, begin capturing audio data by creating a TVILocalAudioTrack
, and begin capturing video by creating a TVILocalVideoTrack
with an associated TVIVideoCapturer
. The iOS Video SDK provides customizable video capturers for both camera and screen capture.
1// Create an audio track2var localAudioTrack = LocalAudioTrack()34// Create a data track5var localDataTrack = LocalDataTrack()67// Create a CameraSource to provide content for the video track8var localVideoTrack : LocalVideoTrack?910// Create a video track with the capturer.11if let camera = CameraSource(delegate: self) {12localVideoTrack = LocalVideoTrack(source: camera)13}
When the client joins a Room, the client can specify which Tracks they wish to share with other Participants. Imagine we want to share the audio and video Tracks we created earlier.
1let connectOptions = ConnectOptions(token: accessToken) { (builder) in2builder.roomName = "my-room"34if let audioTrack = localAudioTrack {5builder.audioTracks = [ audioTrack ]6}7if let dataTrack = localDataTrack {8builder.dataTracks = [ dataTrack ]9}10if let videoTrack = localVideoTrack {11builder.videoTracks = [ videoTrack ]12}13}1415var room = TwilioVideoSDK.connect(options: connectOptions, delegate: self)
For some use cases (such as a ReplayKit broadcast extension) you may wish to connect as a publish-only Participant that is not subscribed to any Tracks. You can disable automatic subscription behavior via ConnectOptions
.
1let connectOptions = ConnectOptions(token: accessToken) { (builder) in2builder.isAutomaticSubscriptionEnabled = false3builder.roomName = "my-room"45if let audioTrack = localAudioTrack {6builder.audioTracks = [ audioTrack ]7}8}910var room = TwilioVideo.connect(options: connectOptions, delegate: self)
When you join a Room, Participants may already be present. You can check for existing Participants in the roomDidConnect:
callback by using the remoteParticipants
getter. To receive RemoteParticipantDelegate
callbacks you will need to set the RemoteParticipant.delegate
property for each connected Remote Participant.
1room = TwilioVideo.connect(options: connectOptions, delegate: self)23// MARK: RoomDelegate45func roomDidConnect(room: Room) {6// The Local Participant7if let localParticipant = room.localParticipant {8print("Local identity \(localParticipant.identity)")910// Set the delegate of the local participant to receive callbacks11localParticipant.delegate = self12}1314// Connected participants already in the room15print("Number of connected Participants \(room.remoteParticipants.count)")1617// Set the delegate of the remote participants to receive callbacks18for remoteParticipant in room.remoteParticipants {19remoteParticipant.delegate = self20}21}2223func participantDidConnect(room: Room, participant: RemoteParticipant) {24print ("Participant \(participant.identity) has joined Room \(room.name)")2526// Set the delegate of the remote participant to receive callbacks27participant.delegate = self28}2930func participantDidDisconnect(room: Room, participant: RemoteParticipant) {31print ("Participant \(participant.identity) has left Room \(room.name)")32}
When Participants connect to or disconnect from a Room that you're connected to, you'll be notified via an event listener. Similar to Room Events, Twilio will fire Participant events if the StatusCallback webhook URL is set when the Room is created. These events help your application keep track of the participants who join or leave a Room.
1// MARK: RoomDelegate23// First, we set a Participant Delegate when a Participant first connects:4func participantDidConnect(room: Room, participant: RemoteParticipant) {5print("Participant connected: \(participant.identity)")6participant.delegate = self7}
To see the Video Tracks being sent by remote Participants, we need to render them to the screen:
1// MARK: RemoteParticipantDelegate23/*4* In the Participant Delegate, we can respond when the Participant adds a Video5* Track by rendering it on screen.6*/7func didSubscribeToVideoTrack(videoTrack: RemoteVideoTrack,8publication: RemoteVideoTrackPublication,9participant: RemoteParticipant) {1011print("Participant \(participant.identity) added a video track.")1213if let remoteView = VideoView.init(frame: self.view.bounds,14delegate:self) {1516videoTrack.addRenderer(remoteView)17self.view.addSubview(remoteView)18self.remoteView = remoteView19}20}2122// MARK: VideoViewDelegate2324// Lastly, we can subscribe to important events on the VideoView25func videoViewDimensionsDidChange(view: VideoView, dimensions: CMVideoDimensions) {26print("The dimensions of the video track changed to: \(dimensions.width)x\(dimensions.height)")27self.view.setNeedsLayout()28}
Sometimes you need to make sure you're looking fantastic before entering a Room. We get it. The iOS SDK provides a means to render a local camera preview outside the context of an active Room:
1// Use CameraSource to produce video from the device's front camera.23if let camera = CameraSource(delegate: self),4let videoTrack = LocalVideoTrack(source: camera) {56// VideoView is a VideoRenderer and can be added to any VideoTrack.7let renderer = VideoView(frame: view.bounds)8}9if let camera = TVICameraCapturer(source: .frontCamera),10let videoTrack = TVILocalVideoTrack(capturer: camera) {1112// TVIVideoView is a TVIVideoRenderer and can be added to any TVIVideoTrack.13let renderer = TVIVideoView(frame: view.bounds)1415// Add renderer to the video track16videoTrack.addRenderer(renderer)1718self.localVideoTrack = videoTrack19self.camera = camera20self.view.addSubview(renderer)21} else {22print("Couldn't create CameraCapturer or LocalVideoTrack")23}
You can disconnect from a Room you're currently participating in. Other Participants will receive a participantDisconnected
event.
1// To disconnect from a Room, we call:2room?.disconnect()34// This results in a callback to RoomDelegate#roomDidDisconnect(room: Room, error: Error?)56// MARK: RoomDelegate78func roomDidDisconnect(room: Room, error: Error?) {9print("Disconnected from room \(room.name)")10}
The Video SDK will raise notifications when a Room is reconnecting due to a network disruption. A Room reconnection is triggered due to a signaling or media reconnection event. To capture when a reconnection is triggered or that it has reconnected:
1// MARK: RoomDelegate23// Error will be either TwilioVideoSDK.Error.signalingConnectionError or TwilioVideoSDK.Error.mediaConnectionError4func roomIsReconnecting(room: Room, error: Error) {5print("Reconnecting to room \(room.name), error = \(String(describing: error))")6}78func roomDidReconnect(room: Room) {9print("Reconnected to room \(room.name)")10}
There are certain instances when an application is put into the background that both the signaling and media connection are closed, which will cause the reconnecting delegate method to be invoked:
The Programmable Video REST API
allows you to control your video applications from your back-end server via HTTP requests. To learn more check out the Programmable Video REST API docs.
We love feedback and questions especially those with helpful debugging information so we can diagnose and respond quickly. When submitting issues or support tickets, it would be great if you add the following:
After gathering the above information, you can get help in a few ways:
For issues related to the Twilio Video iOS SDK itself:
For issues related to the Twilio Video iOS Quickstarts:
To enable debug level logging, add the following code in your application:
TwilioVideoSDK.setLogLevel(.debug)