How to change Float value on images

[pennywise94]

I currently have an application which allows users to edit photos and videos. For the videos, I can simply adjust the value I want using a UISlider, for example I can change the intensity of the Lookup Filter by simply settings the intensity variable - this automatically updates the filter and results in really fast processing.

However, for images, I have been struggling. I currently regenerate the image and assign it to the imageView where I want to display the image. It looks like this:

self.imageSource.removeAllConsumers()
self.metalFilter?.intensity = self.filterValue
self.brightnessFilter?.brightness = self.brightnessValue
DispatchQueue.global(qos: .default).async {
    self.imageSource.add(consumer: self.metalFilter!)
        .add(consumer: self.brightnessFilter!)
        .runSynchronously = true
    self.imageSource.transmitTexture()
    let filteredImage = self.brightnessFilter?.outputTexture?.bb_image
    DispatchQueue.main.async {
        self.imagePreview.image = filteredImage
    }
}

Is this how I should do it? Or could I, as I would wish for, also just update the value - somehow - where the imageView would be updated in real time? I guess this is not possible?

The method I show above used to work with just one Lookup Filter, but now, after adding the brightness filter, and when changing the value using the UISlider, it results in the following error:

-[MTLDebugComputeCommandEncoder setTexture:atIndex:]:396: failed assertion `index(31) must be < 31.'

Any advice on how I can fix/improve this? Tl;dr: I want to change the value of filters (intensity, brightness, contrast) of images and videos using UISlider in real-time.

[pennywise94]

I feel like I'll be able to make it work by using the same method as I use for video. I would set up an imageSource, add my consumers and use a BBMetalView as last consumer. Then, I would transmit the texture using transmitTexture() when needed. Would this be a good solution, or would it consume too much resources? Can it be done more efficiently?

Also, if I go for this approach, how can I save the current MetalView (using the filters I have applied) as an image or get it as UIImage?

[Silence-GitHub]

Setup filter chain first. Add completion callback to the last filter to update image view.

self.imageSource.add(consumer: self.lookupFilter)
    .add(consumer: self.brightnessFilter)
    .addCompletedHandler { [weak self] (_) in // Use weak self or there will be memory leak
        DispatchQueue.main.async {
            guard let self = self else { return }
            self.imageView.image = self.brightnessFilter.outputTexture?.bb_image
        }
}

When changing the slider value, update filter value and transmit texture. Do not change the filter chain here.

self.lookupFilter.intensity = slider.value
self.brightnessFilter.brightness = slider2.value
self.imageSource.transmitTexture()

[pennywise94]

Been trying this, but I have trouble updating the filter chain. I give users the ability to add filters (choose from 11 different LUTs), change brightness, contrast, ... so I remove all consumers when they change a LUT filter, and update the values when they change brightness, contrast etc.

This makes it for me impossible to just update the values and transmitTexture() like you suggest in the last piece of code. Instead, I use a metalView to show the image (just like video), which updates the consumers on the fly. Then, when I save the photo, I initiate a new filter chain with the current LUT (if any) and the values for brightness etc. are hold as variables. This works very well (and fast) now. Is this a suitable approach, specifically when removing and adding new consumers?

[Silence-GitHub]

When changing the slider value, do not change the filter chain (You can setup the filter chain if not exists for the first time slider value changed). When changing the LUT filter, change the filter chain. Just like processing video. I guess the user can not change both LUT filter and slider value at the same time.

[pennywise94]

That's right. But when I use your suggested method, when I present the image in an imageView, the updates don't happen to my image. When I render a metalView, it does render immediately. Is there something I'm missing? Can I, at all, use the metalView or would you not recommend this for images?

[Silence-GitHub]

Your solution is okay. I just think it unnecessary to update the filter chain when changing slider value.

[pennywise94]

You must have understood me wrong. When I change the value using a slider, this is my code:

@objc func changeFilterValue(_ slider: UISlider) {
    filterValue = slider.value / 100
    filterSliderLabel.text = "\(round(slider.value).clean)%"
    self.metalFilter?.intensity = filterValue
    if(imageSource != nil) {
        self.imageSource.transmitTexture()
    }
}

Which works perfectly fine, for images as well as for video. The update happens really fast. It's amazing. I only change the filter change when the user initiates a new lookup filter, like this:

if(imageSource != nil) {
    imageSource.removeAllConsumers()
    metalFilter?.removeAllConsumers()
    brightnessFilter?.removeAllConsumers()
    if(currentFilter == 0) {
        metalFilter = nil
        self.brightnessFilter = BBMetalBrightnessFilter(brightness: self.brightnessValue)
        self.imageSource .add(consumer: brightnessFilter!)
            .add(consumer: metalView)
        self.imageSource.transmitTexture()
    } else {
        self.metalFilter = BBMetalLookupFilter(lookupTable: UIImage(named: "lookup_\(indexPath.item)")!.bb_metalTexture!)
        self.brightnessFilter = BBMetalBrightnessFilter(brightness: self.brightnessValue)
        self.imageSource.add(consumer: brightnessFilter!)
            .add(consumer: metalFilter!)
            .add(consumer: metalView)
        self.imageSource.transmitTexture()
    }
} else if(currentVideoURL != nil) {
    if(!isFiltering) {
        videoSource.removeAllConsumers()
        metalFilter?.removeAllConsumers()
        brightnessFilter?.removeAllConsumers()
        if(indexPath.item == 0) {
            videoSource.add(consumer: metalView)
        } else {
            self.metalFilter = BBMetalLookupFilter(lookupTable: UIImage(named: "lookup_\(indexPath.item)")!.bb_metalTexture!)
            self.brightnessFilter = BBMetalBrightnessFilter(brightness: 0)
            videoSource.add(consumer: brightnessFilter!)
                .add(consumer: metalFilter!)
                .add(consumer: metalView)
        }
    }
}

filterValue = 1
filterSlider.setValue(100, animated: false)
filterSliderLabel.text = "100%"

This works really smooth too. All I still have to deal with now, is the synchronization of video and audio. Have been searching the Internet, looked into GPUImage (which deals with the same issue still, I feel like). Nowhere I can find a solution. But I think I'll just have to go with the temporary solution of using an AVPlayer and let it go out of sync. I'll fix it later when a solution comes up. Amazing library you have provided!