This documentation is for reference only. We are no longer onboarding new customers to Programmable Video. Existing customers can continue to use the product until December 5, 2026.
We recommend migrating your application to the API provided by our preferred video partner, Zoom. We've prepared this migration guide to assist you in minimizing any service disruption.
In this guide, we will show you how to use the VideoSource APIs to share video in a Room. These APIs allow you to choose the built in camera(s), or any other source of content that is available to your application (or extension).
The VideoSource APIs describe producers and consumers of video content. A TVIVideoSource
produces content for a TVILocalVideoTrack
. Sources have the following properties.
A TVIVideoSink
consumes content from a TVIVideoSource
. Sinks have the following properties.
In the next section we will show you how to use the CameraSource API.
A TVICameraSource is a TVIVideoSource that produces content from the built-in cameras. This is probably the first kind of video that you want to share, so it is a good place to begin.
First we want to create a TVICameraSource
, and use that source to create a TVILocalVideoTrack
.
_10guard let cameraSource = CameraSource() else {_10 // Unable to initialize a camera source_10 return_10}_10var videoTrack = LocalVideoTrack(source: cameraSource)
Now that we've setup our Track and Source, its time to start producing frames from one of the built-in cameras. Lets use a TVICameraSource
utility method to help us discover a front facing AVCaptureDevice
.
_10guard let frontCamera = CameraSource.captureDevice(position: .front) else {_10 // The device does not have a front camera._10 return_10}_10_10// Start capturing with the device that we discovered._10cameraSource.startCapture(device: frontCamera)
In this example, TVICameraSource
is automatically determining the best format to capture in. Typically, 640x480 at 30 frames / second is used as the default value.
Next, we want to connect to a Room with the TVILocalVideoTrack
we created earlier.
_10let connectOptions = ConnectOptions(token: accessToken){ (builder) in_10 builder.roomName = "my-room"_10 if let localVideoTrack = self.localVideoTrack {_10 builder.videoTracks = [localVideoTrack]_10 }_10}_10self.room = TwilioVideoSDK.connect(options: connectOptions, delegate: self)
While you can select a single device at start time, TVICameraSource
also supports switching devices while it is already running. For example, you could switch from a front facing device to a rear facing device.
_10guard let rearCamera = CameraSource.captureDevice(position: .back) else {_10 // The device does not have a rear camera._10 return_10}_10_10cameraSource.selectCaptureDevice(rearCamera)
At some point after connecting to a Room, you might decide that you want to stop sharing video from the camera. Start with unpublishing the Track.
_10// Unpublish the Track. We will no longer be sharing video in the Room._10if let participant = self.room?.localParticipant,_10 let videoTrack = self.localVideoTrack {_10 participant.unpublishVideoTrack(videoTrack)_10}
Finally, we will stop the source and destroy the objects.
_10// Stop capturing from the device._10self.camera?.stopCapture(completion: { (error) in_10 if let theError = error {_10 print("Error stopping capture:", theError as Any)_10 }_10_10 self.camera = nil_10 self.localVideoTrack = nil_10})
An AVCaptureDevice
can produce video in many possible formats. TVICameraSource
offers utility methods to discover formats that are suitable for video streaming. Consider executing the following code on your iOS device:
_10// Assume that we discovered "frontDevice" earlier._10_10let formats = CameraSource.supportedFormats(captureDevice: frontDevice)_10print(formats)
When this code is run on an iPhone X with iOS 12.4, the following formats are returned.
Dimensions | Frame Rate | Pixel Format |
---|---|---|
192 x 144 | 30 | 420f |
352 x 288 | 30 | 420f |
480 x 360 | 30 | 420f |
640 x 480 | 30 | 420f |
960 x 540 | 30 | 420f |
1280 x 720 | 30 | 420f |
1920 x 1080 | 30 | 420f |
1920 x 1440 | 30 | 420f |
3088 x 2320 | 30 | 420f |
Once you've determined which format you would like to use, you can provide it when starting capture.
_10// Formats are ordered by increasing dimensions. Start with the smallest size._10cameraSource.startCapture(device: frontDevice,_10 format: formats.firstObject as! VideoFormat,_10 completion: nil)
In some applications, it may be important to change formats at runtime with as little disruption to the camera feed as possible.
_10// Select another format for the front facing camera._10cameraSource.selectCaptureDevice(frontDevice,_10 format: formats.lastObject as! VideoFormat,_10 completion: nil)
Device formats afford quite a lot of flexibility, but there are some cases that AVCaptureDevice
does not support out of the box. For example, what if you wanted to:
These are both cases where you want to publish video in a different aspect ratio or size than AVCaptureDevice
can produce. That is okay, because format requests are here to help with this problem.
_28let frontDevice = CameraSource.captureDevice(position: .front)!_28let formats = CameraSource.supportedFormats(captureDevice: frontDevice)_28_28// We match 640x480 directly, since it is known to be supported by all devices._28var preferredFormat: VideoFormat?_28for format in formats {_28 let theFormat = format as! VideoFormat_28 if theFormat.dimensions.width == 640,_28 theFormat.dimensions.height == 480 {_28 preferredFormat = theFormat_28 }_28}_28_28guard let captureFormat = preferredFormat else {_28 // The preferred format could not be found._28 return_28}_28_28// Request cropping to 480x480._28let croppingRequest = VideoFormat()_28let dimension = captureFormat.dimensions.height_28croppingRequest.dimensions = CMVideoDimensions(width: dimension,_28 height: dimension)_28_28self.camera?.requestOutputFormat(croppingRequest)_28self.camera?.startCapture(device: frontDevice,_28 format: captureFormat,_28 completion: nil)
The following diagram shows the effect of a format request on frames produced by TVICameraSource
.
Take a look at the iOS QuickStart Example to learn more about using TVICameraSource
.
The TVICameraSource
provides flexibility in how it tracks video orientation for capture and preview. By default, the TVICameraSource
monitors -[UIApplication statusBarOrientation]
for orientation changes. With the addition of the UIWindowScene
APIs in iOS 13, TVICameraSource
now has a property, TVICameraSourceOptions.orientationTracker
, which allows you to specify how the TVICameraSource
should track orientation changes.
The orientationTracker
property accepts an object that implements the TVICameraSourceOrientationTracker
protocol. A default implementation, TVIUserInterfaceTracker
is provided with the SDK. TVIUserInterfaceTracker
monitors for changes in UIInterfaceOrientation
at the application or scene level. For example, if you wish to track orientation changes based on a scene, you would provide the scene to track when creating the TVICameraSourceOptions
.
_10// Track the orientation of the key window's scene._10let options = CameraSourceOptions { (builder) in_10 if let keyScene = UIApplication.shared.keyWindow?.windowScene {_10 builder.orientationTracker = UserInterfaceTracker(scene: keyScene)_10 }_10}_10let camera = CameraSource(options: options, delegate: self)
You will also need to forward UIWindowScene
events from your UIWindowSceneDelegate
to keep TVIUserInterfaceTracker
up to date as the scene changes.
_10// Forward UIWindowScene events_10func windowScene(_ windowScene: UIWindowScene,_10 didUpdate previousCoordinateSpace: UICoordinateSpace,_10 interfaceOrientation previousInterfaceOrientation: UIInterfaceOrientation,_10 traitCollection previousTraitCollection: UITraitCollection) {_10 UserInterfaceTracker.sceneInterfaceOrientationDidChange(windowScene)_10}
You can also manually control how orientation is tracked. For example, you might decide to use UIDevice
instead of UIScene
to determine the orientation of the camera. To do this, you would create your own implementation of TVICameraSourceOrientationTracker
which invokes the - (void)trackerOrientationDidChange:(AVCaptureVideoOrientation)orientation
callback method when the device's orientation changes.
VideoSources are real-time producers of content. Importantly, to optimize for low latency delivery of individual frames is not guaranteed. The video pipeline continuously monitors network and device conditions and may respond by:
If you would like to implement your own VideoSource, or learn about advanced usage of TVICameraSource
then an excellent place to begin is with our sample code.