goal c – iOS Digicam Freezes and Crashes with MLKit Barcode Scanning in Kotlin Multiplatform

I am growing an app utilizing Kotlin Multiplatform and Googles MLKit for barcode scanning. I am unable to create a VisionImage within the captureOutput technique of a AVCaptureVideoDataOutputSampleBufferDelegate. When making an attempt, I’m encountering periodic freezes each 15 frames for 10 seconds.

In my CameraViewController (iosMain)

class CameraViewController(
    non-public val onQRCodeDetected: (String) -> Unit
) : UIViewController(null, null), AVCaptureVideoDataOutputSampleBufferDelegateProtocol {

I overwrite captureOutput

    override enjoyable captureOutput(
        output: AVCaptureOutput,
        didOutputSampleBuffer: CMSampleBufferRef?,
        fromConnection: AVCaptureConnection
    ) {        
        frameCounter+=1
        logging("CViewC").i { "FRAME #$frameCounter" }

to date every part works good. captureOutput will get referred to as every body in time and logs the framecount. Once I add the next line issues break.

val visionImage = MLKVisionImage(didOutputSampleBuffer)

Now the primary 15 calles of captureOutput are okay. Throughout this period the preview on the cellphone works fluent. Then the preview freezes and captureOutput will not be referred to as. This lasts for 10 seconds. After the ten seconds I obtain once more 15 frames usually. This cicle goes on and the app by no means crashes.

Once I add now the road

CFRelease(didOutputSampleBuffer)

I get a steady move of 288 frames till a crash. The crash report signifies that I attempted to launch an already launched buffer(Detected over-release of a CFTypeRef %p (%lu / %s)).

I attempted to repeat the Buffer:

val copiedBufferVar = nativeHeap.alloc()
CMSampleBufferCreateCopy(
    allocator = kCFAllocatorDefault,
    sbuf = didOutputSampleBuffer,
    sampleBufferOut = copiedBufferVar.ptr
)
val visionImage = MLKVisionImage(didOutputSampleBuffer.worth)
CFRelease(copiedBufferVar.worth)

Since I assumed the Buffers weren’t launched from reminiscence due to the sturdy reference of the VisionImage to the Buffer. (I do not know if that is smart – I desperately tried to make sense of one thing). This resulted in the identical factor. A cycle of 15 frames OK and 10 Seconds freeze.

I attempted to launch the VisionImage from reminiscence by making it nullable and setting it to null after utilization. This additionally change nothing. LLM’s really useful utilizing a autoreleasepool however that additionally did not assist.

So I suppose it’s a reminiscence downside. I assumed my AVCaptureSession takes care of these items robotically. I configured it like this. I eliminated some null checks to maintain it shorter right here.

@OptIn(ExperimentalForeignApi::class)
non-public enjoyable setupCaptureSession() {

    captureSession = AVCaptureSession().apply {
        beginConfiguration()
        sessionPreset = AVCaptureSessionPreset1280x720
    }

    val system = AVCaptureDevice.defaultDeviceWithMediaType(AVMediaTypeVideo)
    val enter = AVCaptureDeviceInput.deviceInputWithDevice(system, null)
    if (captureSession?.canAddInput(enter) == true) {
        captureSession?.addInput(enter)
    }
    videoOutput = AVCaptureVideoDataOutput().apply {
        setSampleBufferDelegate(this@CameraViewController, processingQueue)
        videoSettings = mapOf(
            kCVPixelBufferPixelFormatTypeKey to NSNumber(kCVPixelFormatType_32BGRA) 
        )
        alwaysDiscardsLateVideoFrames = true

    }
    if (captureSession?.canAddOutput(videoOutput!!) == true) {
        captureSession?.addOutput(videoOutput!!)
    }
    captureSession?.commitConfiguration()
    previewLayer = AVCaptureVideoPreviewLayer(session = captureSession!!).apply {
        videoGravity = AVLayerVideoGravityResizeAspectFill
    }
}