Live Text API in iOS 16 — Scanning Data With the Camera in SwiftUI | by joker hook | Jun, 2022

Photo by Markus Spiske on Unsplash

WWDC22 brings brilliant Live Text data scanning tools which let users scan text and codes with the camera, similar to the Live Text interface in the Camera app for iOS and iPadOS developers.

In this article, I will focus on the new API called DataScannerViewController and share my experience with how to embed this API into your SwiftUI code. The following photo shows today’s demo.

Demo app that we will create

Because this demo app can only be running on real-world devices, so to access users’ cameras you’d better provide a clear statement of why you need to access their cameras.

You provide the reason for using the camera in the Xcode project configuration. Add the NSCameraUsageDescription key to the target’s Information Property List in Xcode.

The following steps I provide come from Apple’s official document: Scanning data with the camera.

  1. In the project editor, select the target and click Info.
  2. Under Custom iOS Target Properties, click the Plus button in any row.
  3. From the pop-up menu in the Key column, choose Privacy — Camera Usage Description.
  4. In the Value column, enter the reason, such as “Your camera is used to scan text and codes.”

In your ContentView.swift file, add the following code:

VStack {
Text(scanResults)
.padding()
Button {
// Enable Scan Document Action
} label: {
Text("Tap to Scan Documents")
.foregroundColor(.white)
.frame(width: 300, height: 50)
.background(Color.blue)
.cornerRadius(10)
}
}

Where scanResult is a String variable which represents the camera scan result that will use to illustrate what the camera sees during scanning.

@State private var scanResults: String = ""

The button here is used to present the scanning view.

When someone taps the button, the device will be ready to scan data. However, not all devices support this function. Or even when the device supports scan data, when the user denies providing the camera usage permission, things may fail when tapping the button.

In such cases, I will provide an alert view that shows a message when the device is not capable of scanning data.

@State private var showDeviceNotCapacityAlert = false

The code above provides a variable to choose whether shows the alert view or not. If showDeviceNotCapacityAlert is truethen shows the alert view.

Add the following code behind the VStack code:

.alert("Scanner Unavailable", isPresented: $showDeviceNotCapacityAlert, actions: {})

Finally, when the device is ready for scanning data, we need to present a scanning view, like the above code, and add the following code to your ContentView.swift file:

@State private var showCameraScannerView = falsevar body: some View {
VStack {
...
}
.sheet(isPresented: $showCameraScannerView) {
// Present the scanning view
}
...
}

Now thing what we left is when we tap the button, if the device is not capable of scanning data, an alert view will show. We use isDeviceCapacity to check whether the device can use this function or not.

@State private var isDeviceCapacity = false

Now add the following code inside the Button action:

if isDeviceCapacity {
self.showCameraScannerView = true
} else {
self.showDeviceNotCapacityAlert = true
}

Create a new swift file and named it CameraScanner.swift . Add the following code here:

struct CameraScanner: View {
@Environment(.presentationMode) var presentationMode
var body: some View {
NavigationView {
Text("Scanning View")
.toolbar {
ToolbarItem(placement: .navigationBarLeading) {
Button {
self.presentationMode.wrappedValue.dismiss()
} label: {
Text("Cancel")
}
}
}
.interactiveDismissDisabled(true)
}
}
}

When the app is opened by users, we need to check whether the scanner is available or not. Simply add the following code then this app is opened:

var body: some View {
VStack {
...
}
.onAppear {
isDeviceCapacity = (DataScannerViewController.isSupported && DataScannerViewController.isAvailable)
}
...
}

Only if the above convenience property that checks both values ​​returns true we can open the scanning view.

To implement a view controller that can be used in a SwiftUI View, first of all we need to use UIViewControllerRepresentable to wrap a UIKit view controller so that it can be used inside SwiftUI. Create a new swift file named CameraScannerViewController.swift and simply add the following code here:

import SwiftUI
import UIKit
import VisionKit
struct CameraScannerViewController: UIViewControllerRepresentable {
func makeUIViewController(context: Context) -> DataScannerViewController {
let viewController = DataScannerViewController(recognizedDataTypes: [.text()],qualityLevel: .fast,recognizesMultipleItems: false, isHighFrameRateTrackingEnabled: false, isHighlightingEnabled: true)
return viewController
}
func updateUIViewController(_ viewController: DataScannerViewController, context: Context) {}}

The above code returns a view controller which provides the interface for scanning items in the live video. In this article, I will only focus on scanning text data so the recognizedDataTypes here only contains .text() property.

After creating the view controller and before we present it, set its delegate to an object in this app that handles the DataScannerViewControllerDelegate protocol callbacks.

In UIKit it’s easy to write the following code:

viewController.delegate = self

Luckily, it can be very convenient to handle the DataScannerViewControllerDelegate in SwiftUI.

SwiftUI’s coordinators are designed to act as delegates for UIKit view controllers. Remember, “delegates” are objects that respond to events that occur elsewhere. For example, UIKit lets us attach a delegate object to its text field view, and that delegate will be notified when the user types anything, when they press return, and so on. This meant that UIKit developers could modify the way their text field behaved without having to create a custom text field type of their own.

So add the following code inside the CameraScannerViewController:

func makeCoordinator() -> Coordinator {
Coordinator(self)
}
class Coordinator: NSObject, DataScannerViewControllerDelegate {
var parent: CameraScannerViewController
init(_ parent: CameraScannerViewController) {
self.parent = parent
}
}

Now we can use the similar code inside makeUIViewController just on the top of the return code:

func makeUIViewController(context: Context) -> DataScannerViewController {
...
viewController.delegate = context.coordinator
return viewController
}

It’s time to start the data scanning. Once the user allows access to the camera without restrictions, you can begin scanning for items that appear in the live video by invoking the startScanning() method. In this case, when the scanning view is presented we will need the view to perform the scanning action.

We need a value to tell the scanning view to scan, add the following code to the CameraScannerViewController :

@Binding var startScanning: Bool

When the startScanning‘s value is set to true, we need to update UIViewController and start scanning, add the following code inside updateUIViewController :

func updateUIViewController(_ viewController: DataScannerViewController, context: Context) {
if startScanning {
try? viewController.startScanning()
} else {
viewController.stopScanning()
}
}

When we tap a recognized item in the live video, the view controller invokes the dataScanner(_:didTapOn:) delegate method and passes the recognized item. Implement this method to take some action depending on the item the user taps. Use the parameters of the RecognizedItem enum to get details about the item, such as the bounds.

In this case, to handle when we tap a text that the camera recognized, implement the dataScanner(_:didTapOn:) method to perform an action that shows the result in the screen. So add the following code inside Coordinator class:

class Coordinator: NSObject, DataScannerViewControllerDelegate {
...
func dataScanner(_ dataScanner: DataScannerViewController, didTapOn item: RecognizedItem) {
switch item {
case .text(let text):
parent.scanResult = text.transcript
default:
break
}
}
}

And add a Binding property inside CameraScannerViewController :

@Binding var scanResult: String

It’s time to update the CameraScanner.swift file. Simply add the following code:

@Binding var startScanning: Bool
@Binding var scanResult: String

And change the Text(“Scanning View”) to the following code:

var body: some View {
NavigationView {
CameraScannerViewController(startScanning: $startScanning, scanResult: $scanResult)
...
}
}

Finally, add the following code to ContentView :

struct ContentView: View {
...
@State private var scanResults: String = ""
var body: some View {
VStack {
...
}
.sheet(isPresented: $showCameraScannerView) {
CameraScanner(startScanning: $showCameraScannerView, scanResult: $scanResults)
}
...
}
}

scanResults is used to pass the value through views. Once the camera scan something, scanResults will be updated and then update the Text view.

Now run this project and enjoy yourself.

You can find the source code on Github.

Leave a Comment