Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 11 additions & 7 deletions TRIAGE.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ Legend:
| #390 | not a bug | Reports `start` / `end` time behavior for video compression, but the current public video API does not expose trim parameters. |
| #387 | needs info | Gradle binary store corruption looks environment-specific; report does not isolate a library code change. |
| #384 | needs info | Performance question, not a reproducible defect report. |
| #383 | real | Android transcode pipeline can blow up on pathological audio metadata (`uint32 overflow`). This branch improves failure handling so it rejects instead of silently succeeding. |
| #383 | real, fixed here | Android transcode pipeline can blow up on pathological audio metadata (`uint32 overflow`). This branch skips unsupported copied audio metadata instead of crashing. |
| #382 | needs info | “Works in dev, fails in prod” has no logs or repro app. |
| #381 | feature | Nitro Modules migration request. |
| #380 | real, fixed here | Android manual compression could produce invalid tiny files when `maxSize` generated odd dimensions or invalid output. This branch normalizes dimensions and rejects invalid output files. |
Expand All @@ -25,26 +25,26 @@ Legend:
| #375 | real, fixed here | Quality complaint is consistent with the old hard bitrate cap. Adaptive bitrate selection in this branch directly targets it. |
| #371 | duplicate | Likely another Android video transcode failure in the same cluster as #343 / #380 / #376. |
| #370 | stale | Current tree no longer imports `AssetsLibrary`; this is already gone. |
| #369 | real | “Playable only in VLC” is credible output-container compatibility fallout; likely same Android transcode/output-validation cluster as #380 / #376. |
| #369 | real, fixed here | “Playable only in VLC” is credible output-container compatibility fallout. This branch fast-starts Android MP4 outputs and skips unsupported audio sample metadata that can produce incompatible containers. |
| #367 | stale | Same `AssetsLibrary` removal request as #370 / #362; already addressed in current sources. |
| #366 | real | `libandroidlame.so` 16 KB page-size warning is a real Android dependency issue, but separate from video compression. |
| #366 | real, fixed here | `libandroidlame.so` 16 KB page-size warning is addressed by the current `TAndroidLame` fork dependency already present in this tree. |
| #365 | real, fixed here | Android parsed bitrate metadata as `Int` and could overflow on bogus sentinel values. This branch now clamps metadata safely. |
| #364 | real | Manual compression crash report is credible; likely same manual-path sizing/metadata weaknesses addressed here, but no sample was attached. |
| #364 | real, fixed here | Manual compression crash report is credible; manual-path sizing, metadata, output validation, and audio-container hardening in this branch address the likely causes. |
| #363 | real, fixed here | iOS assumed a video track existed and could crash on audio-only MP4 files. This branch now guards that path. |
| #362 | stale | Another `AssetsLibrary` build failure that no longer matches the current tree. |
| #358 | feature | Live photo optimization request. |
| #356 | real, fixed here | Android AGP 8+ `BuildConfig` generation issue. This branch enables `buildConfig` in the library Gradle file. |
| #354 | stale | Old Android build failure references the previous `AndroidLame-kotlin` dependency coordinates, which are no longer in this tree. |
| #353 | feature | Audio speed-up request. |
| #352 | real | Thumbnail generation failing on some videos is plausible and has a sample, but was not investigated in this pass. |
| #352 | real, fixed here | Thumbnail generation now retries with tolerant frame extraction and reports a deterministic error when no frame can be decoded. |
| #348 | stale | Report targets `1.11.0` Gradle sync behavior with minimal details; no matching current-tree defect was found. |
| #347 | real | Image quality parameter complaint is credible and independent of the video work in this branch. |
| #347 | real, fixed here | Image quality is now clamped consistently before JPEG encoding on Android and iOS. |
| #345 | stale | Current tree has only one TurboModule spec (`src/Spec/NativeCompressor.ts`); the duplicate-spec issue no longer matches HEAD. |
| #343 | real, fixed here | Repeated 4k Android compression failures line up with old manual sizing/bitrate behavior. This branch reworks the compression profile for high-res inputs. |
| #318 | stale | Old dependency-resolution issue references outdated dependency coordinates and repository/network failures. |
| #308 | duplicate | Broad “sometimes compresses, sometimes not” report fits the Android video-quality/output cluster but lacks a repro sample. |
| #302 | needs info | Slow compression is a product concern, but the report is only a timing complaint with no reproducible defect. |
| #263 | real | iOS background upload returning an empty response body is a credible platform-specific bug outside this video-focused change set. |
| #263 | real, fixed here | iOS background upload now accumulates response data and returns the response body string like Android. |

## Main clusters

Expand Down Expand Up @@ -85,4 +85,8 @@ These should be closed upstream unless a current repro still exists on the lates
- Android: enable `buildConfig` generation for AGP 8+ builds
- Android: clamp metadata parsing and reject invalid transcode output
- Android: adaptive video compression profile for high-resolution inputs
- Android: fast-start compressed MP4 outputs and skip unsupported copied audio sample metadata
- Android/iOS: clamp image and thumbnail JPEG quality values
- Android/iOS: harden thumbnail frame extraction for difficult source videos
- iOS: guard missing video tracks and use the same adaptive sizing/bitrate strategy
- iOS: return background-upload response bodies consistently
Original file line number Diff line number Diff line change
Expand Up @@ -113,16 +113,17 @@ object ImageCompressor {

fun compress(image: Bitmap?, output: ImageCompressorOptions.OutputType, quality: Float,disablePngTransparency:Boolean): ByteArrayOutputStream {
var stream = ByteArrayOutputStream()
val normalizedQuality = Math.round(100 * quality.coerceIn(0f, 1f))
if (output === ImageCompressorOptions.OutputType.jpg)
{
image!!.compress(CompressFormat.JPEG, Math.round(100 * quality), stream)
image!!.compress(CompressFormat.JPEG, normalizedQuality, stream)
}
else
{
var bitmap = image
if(disablePngTransparency)
{
image!!.compress(CompressFormat.JPEG, Math.round(100 * quality), stream)
image!!.compress(CompressFormat.JPEG, normalizedQuality, stream)
val byteArray: ByteArray = stream.toByteArray()
stream=ByteArrayOutputStream()
bitmap = BitmapFactory.decodeByteArray(byteArray, 0, byteArray.size)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ class CreateVideoThumbnailClass(private val reactContext: ReactApplicationContex
}

val headers: Map<String, String> = if (options.hasKey("headers")) options.getMap("headers")!!.toHashMap() as Map<String, String> else HashMap<String, String>()
val quality = if (options.hasKey("quality")) (options.getDouble("quality") * 100).toInt().coerceIn(0, 100) else DEFAULT_THUMBNAIL_QUALITY
val fileName = if (TextUtils.isEmpty(cacheName)) "thumb-" + UUID.randomUUID().toString() else "$cacheName.$format"
var fOut: OutputStream? = null

Expand All @@ -73,7 +74,7 @@ class CreateVideoThumbnailClass(private val reactContext: ReactApplicationContex
fOut = FileOutputStream(file)

// 100 means no compression, the lower you go, the stronger the compression
image.compress(Bitmap.CompressFormat.JPEG, 90, fOut)
image.compress(Bitmap.CompressFormat.JPEG, quality, fOut)
fOut.flush()
fOut.close()

Expand All @@ -96,6 +97,8 @@ class CreateVideoThumbnailClass(private val reactContext: ReactApplicationContex
}

companion object {
private const val DEFAULT_THUMBNAIL_QUALITY = 90

// delete previously added files one by one untill requred space is available
fun clearCache(cacheDir: String?,promise:Promise, reactContext: ReactApplicationContext) {
val cacheDirectory=cacheDir?.takeIf { it.isNotEmpty() } ?:"/thumbnails"
Expand Down Expand Up @@ -134,29 +137,47 @@ class CreateVideoThumbnailClass(private val reactContext: ReactApplicationContex
}

private fun getBitmapAtTime(context: Context?, filePath: String?, time: Int, headers: Map<String, String>): Bitmap {
check(!filePath.isNullOrEmpty()) { "Video file path cannot be null or empty" }
val retriever = MediaMetadataRetriever()
if (URLUtil.isFileUrl(filePath)) {
val decodedPath: String?
decodedPath = try {
URLDecoder.decode(filePath, "UTF-8")
} catch (e: UnsupportedEncodingException) {
filePath
}
retriever.setDataSource(decodedPath!!.replace("file://", ""))
} else if (filePath!!.contains("content://")) {
retriever.setDataSource(context, Uri.parse(filePath))
} else {
check(Build.VERSION.SDK_INT >= 14) { "Remote videos aren't supported on sdk_version < 14" }
retriever.setDataSource(filePath, headers)
}
val image = retriever.getFrameAtTime((time * 1000).toLong(), MediaMetadataRetriever.OPTION_CLOSEST_SYNC)
try {
retriever.release()
} catch (e: IOException) {
throw RuntimeException(e)
if (URLUtil.isFileUrl(filePath)) {
val decodedPath: String? = try {
URLDecoder.decode(filePath, "UTF-8")
} catch (e: UnsupportedEncodingException) {
filePath
}
retriever.setDataSource(decodedPath!!.replace("file://", ""))
} else if (filePath.contains("content://")) {
retriever.setDataSource(context, Uri.parse(filePath))
} else {
check(Build.VERSION.SDK_INT >= 14) { "Remote videos aren't supported on sdk_version < 14" }
retriever.setDataSource(filePath, headers)
}

val requestedTimeUs = (time * 1000).toLong()
val frameAttempts = arrayOf(
Pair(requestedTimeUs, MediaMetadataRetriever.OPTION_CLOSEST_SYNC),
Pair(requestedTimeUs, MediaMetadataRetriever.OPTION_CLOSEST),
Pair(1_000_000L, MediaMetadataRetriever.OPTION_CLOSEST_SYNC),
)
for ((timeUs, option) in frameAttempts) {
val image = try {
retriever.getFrameAtTime(timeUs, option)
} catch (e: RuntimeException) {
null
}
if (image != null) {
return image
}
}
error("Unable to extract video frame from file")
} finally {
try {
retriever.release()
} catch (e: IOException) {
// Ignore
}
}
checkNotNull(image) { "File doesn't exist or not supported" }
return image
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -41,10 +41,21 @@ object Compressor {
private const val INVALID_BITRATE =
"The provided bitrate is smaller than what is needed for compression, " +
"try to set isMinBitRateEnabled to false"
private val SUPPORTED_AUDIO_SAMPLE_RATES = setOf(
8000, 11025, 12000, 16000, 22050, 24000, 32000, 44100, 48000, 64000
)
private const val STREAMABLE_SUFFIX = "-streamable"
private const val DEFAULT_OUTPUT_EXTENSION = "mp4"

// Flag to check if compression is running
var isRunning = true

private fun getStreamableOutputFile(cacheFile: File): File =
File(
cacheFile.parentFile ?: File("."),
"${cacheFile.nameWithoutExtension}$STREAMABLE_SUFFIX.${cacheFile.extension.ifEmpty { DEFAULT_OUTPUT_EXTENSION }}"
)

suspend fun compressVideo(
index: Int,
context: Context,
Expand Down Expand Up @@ -417,18 +428,27 @@ object Compressor {

var resultFile = cacheFile

// Process the result and create a streamable video if requested
streamableFile?.let {
try {
val result = StreamableVideo.start(`in` = cacheFile, out = File(it))
resultFile = File(it)
if (result && cacheFile.exists()) {
try {
// Keep default outputs browser-compatible by moving the MP4 metadata before media data.
val targetFile = streamableFile?.let { File(it) } ?: getStreamableOutputFile(cacheFile)
val outputFile = if (targetFile.absolutePath == cacheFile.absolutePath) {
getStreamableOutputFile(cacheFile)
} else {
targetFile
}
val result = StreamableVideo.start(`in` = cacheFile, out = outputFile)
if (result) {
if (streamableFile == null || targetFile.absolutePath == cacheFile.absolutePath) {
cacheFile.delete()
outputFile.renameTo(cacheFile)
resultFile = cacheFile
} else {
resultFile = outputFile
cacheFile.delete()
}

} catch (e: Exception) {
printException(e)
}
} catch (e: Exception) {
printException(e)
}
if (!resultFile.exists() || resultFile.length() <= 32) {
return Result(
Expand Down Expand Up @@ -464,8 +484,16 @@ object Compressor {
if (audioIndex >= 0 && !disableAudio) {
extractor.selectTrack(audioIndex)
val audioFormat = extractor.getTrackFormat(audioIndex)
if (!isSupportedAudioFormat(audioFormat)) {
extractor.unselectTrack(audioIndex)
return
}
val muxerTrackIndex = mediaMuxer.addTrack(audioFormat, true)
var maxBufferSize = audioFormat.getInteger(MediaFormat.KEY_MAX_INPUT_SIZE)
var maxBufferSize = if (audioFormat.containsKey(MediaFormat.KEY_MAX_INPUT_SIZE)) {
audioFormat.getInteger(MediaFormat.KEY_MAX_INPUT_SIZE)
} else {
64 * 1024
}

if (maxBufferSize <= 0) {
maxBufferSize = 64 * 1024
Expand Down Expand Up @@ -508,6 +536,16 @@ object Compressor {
}
}

private fun isSupportedAudioFormat(audioFormat: MediaFormat): Boolean {
if (!audioFormat.containsKey(MediaFormat.KEY_SAMPLE_RATE) ||
!audioFormat.containsKey(MediaFormat.KEY_CHANNEL_COUNT)) {
return false
}
val sampleRate = audioFormat.getInteger(MediaFormat.KEY_SAMPLE_RATE)
val channelCount = audioFormat.getInteger(MediaFormat.KEY_CHANNEL_COUNT)
return channelCount > 0 && sampleRate in SUPPORTED_AUDIO_SAMPLE_RATES
}

// Function to prepare the video encoder
private fun prepareEncoder(outputFormat: MediaFormat, hasQTI: Boolean): MediaCodec {

Expand Down
5 changes: 3 additions & 2 deletions ios/Image/ImageCompressor.swift
Original file line number Diff line number Diff line change
Expand Up @@ -167,13 +167,14 @@ class ImageCompressor {
static func writeImage(_ image: UIImage, output: Int, quality: Float, outputExtension: String, isBase64: Bool, disablePngTransparency: Bool, isEnableAutoCompress: Bool, actualImagePath: String?)-> String {
var data: Data
var exception: NSException?
let normalizedQuality = CGFloat(min(max(quality, 0), 1))

switch OutputType(rawValue: output)! {
case .jpg:
data = image.jpegData(compressionQuality: CGFloat(quality))!
data = image.jpegData(compressionQuality: normalizedQuality)!
case .png:
if disablePngTransparency {
data = image.jpegData(compressionQuality: CGFloat(quality))!
data = image.jpegData(compressionQuality: normalizedQuality)!
let compressedImage = UIImage(data: data)
data = compressedImage!.pngData()!
} else {
Expand Down
61 changes: 40 additions & 21 deletions ios/Utils/CreateVideoThumbnail.swift
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import AVFoundation
import UIKit

class CreateVideoThumbnail: NSObject {
private static let defaultQuality = 0.9

func create(_ fileUrl:String, options: NSDictionary, resolve: @escaping RCTPromiseResolveBlock, rejecter reject: @escaping RCTPromiseRejectBlock) {
let headers = options["headers"] as? [String: Any] ?? [:]
Expand Down Expand Up @@ -49,21 +50,26 @@ class CreateVideoThumbnail: NSObject {
vidURL = URL(fileURLWithPath: fileUrl)
}

let asset = AVURLAsset(url: vidURL!, options: ["AVURLAssetHTTPHeaderFieldsKey": headers])
guard let vidURL = vidURL else {
reject("CreateVideoThumbnail", "Unable to create a URL from the provided video path", nil)
return
}
let quality = Self.normalizedQuality(options["quality"])
let asset = AVURLAsset(url: vidURL, options: ["AVURLAssetHTTPHeaderFieldsKey": headers])
generateThumbImage(asset: asset, atTime: 0, completion: { thumbnail in
// Generate thumbnail
var data: Data? = thumbnail.jpegData(compressionQuality: 1.0)

if let data = data {
try? data.write(to: URL(fileURLWithPath: fullPath))
resolve([
"path": fullPath,
"size": Float(data.count),
"mime": "image/\(format)",
"width": Float(thumbnail.size.width),
"height": Float(thumbnail.size.height)
] as [String : Any])
guard let data = thumbnail.jpegData(compressionQuality: quality) else {
reject("CreateVideoThumbnail", "Unable to encode video thumbnail", nil)
return
}
try? data.write(to: URL(fileURLWithPath: fullPath))
resolve([
"path": fullPath,
"size": Float(data.count),
"mime": "image/\(format)",
"width": Float(thumbnail.size.width),
"height": Float(thumbnail.size.height)
] as [String : Any])
}, failure: { error in
reject(error._domain, error.localizedDescription, nil)
})
Expand Down Expand Up @@ -98,20 +104,33 @@ class CreateVideoThumbnail: NSObject {
}
}

private static func normalizedQuality(_ value: Any?) -> CGFloat {
let rawValue = (value as? NSNumber)?.doubleValue ?? defaultQuality
return CGFloat(min(max(rawValue, 0), 1))
}

func generateThumbImage(asset: AVURLAsset, atTime timeStamp: Int, completion: @escaping (UIImage) -> Void, failure: @escaping (Error) -> Void) {
let generator = AVAssetImageGenerator(asset: asset)
generator.appliesPreferredTrackTransform = true
generator.maximumSize = CGSize(width: 512, height: 512)
generator.requestedTimeToleranceBefore = CMTimeMake(value: 0, timescale: 1000)
generator.requestedTimeToleranceAfter = CMTimeMake(value: 0, timescale: 1000)
let time = CMTimeMake(value: Int64(timeStamp), timescale: 1000)
generator.generateCGImagesAsynchronously(forTimes: [NSValue(time: time)]) { _, image, _, result, error in
if result == .succeeded, let cgImage = image {
let thumbnail = UIImage(cgImage: cgImage)
completion(thumbnail)
} else if let error = error {
failure(error)
generator.requestedTimeToleranceBefore = .positiveInfinity
generator.requestedTimeToleranceAfter = .positiveInfinity
let times = [
CMTimeMake(value: Int64(timeStamp), timescale: 1000),
CMTimeMake(value: 1000, timescale: 1000)
]
var lastError: Error?

for time in times {
do {
let cgImage = try generator.copyCGImage(at: time, actualTime: nil)
completion(UIImage(cgImage: cgImage))
return
} catch {
lastError = error
}
}

failure(lastError ?? NSError(domain: "CreateVideoThumbnail", code: -1, userInfo: [NSLocalizedDescriptionKey: "Unable to create thumbnail"]))
}
}
Loading
Loading