libc++abi: terminating due to uncaught exception of type std::runtime_error: [METAL] Command buffer execution failed: Invalid Resource (00000009:kIOGPUCommandBufferCallbackErrorInvalidResource)

[jrp2014]

Probably an MLX bug or bugs in the recent versions of mlx, but you may be better placed to give them a pointer as to where the problem lines:

python check_models.py -v --debug
AttributeError: 'MessageFactory' object has no attribute 'GetPrototype'
AttributeError: 'MessageFactory' object has no attribute 'GetPrototype'
AttributeError: 'MessageFactory' object has no attribute 'GetPrototype'
AttributeError: 'MessageFactory' object has no attribute 'GetPrototype'
AttributeError: 'MessageFactory' object has no attribute 'GetPrototype'

MLX version: 0.24.0
MLX-VLM version: 0.1.21


Processing file: /Users/jrp/Pictures/Processed/20250315-163157_DSC02909_openWith-1 copy.jpg
Image dimensions: 7992x5853 (46.8 MPixels)
Lat & Longitude: Unknown location
Date:            2025-03-15 16:31:57
Description:     Castle Shopping Centre, Banbury, England, United Kingdom, UK

vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
Processing HuggingFaceTB/SmolVLM-Instruct
Fetching 12 files: 100%|█████████████████████████████████████████████████████████████| 12/12 [00:00<00:00, 17829.13it/s]
Fetching 12 files: 100%|█████████████████████████████████████████████████████████████| 12/12 [00:00<00:00, 23899.17it/s]
==========
Files: ['/Users/jrp/Pictures/Processed/20250315-163157_DSC02909_openWith-1 copy.jpg'] 

Prompt: <|im_start|>User:<image>Provide a factual caption, description and comma-separated keywords or tags for this image so that it can be catalogued and searched for easily. The picture was taken in Castle Shopping Centre, Banbury, England, United Kingdom, UK on 2025-03-15 16:31:57<end_of_utterance>
Assistant:
 Castle Shopping Centre, Banbury, England, United Kingdom, UK, 2025-03-15 16:31:57
==========
Prompt: 1259 tokens, 1080.214 tokens-per-sec
Generation: 35 tokens, 120.016 tokens-per-sec
Peak memory: 5.443 GB

╔═══════════════════╦═══════════╦═══════════╗
║ Memory usage (MB) ║    Pre    ║   Post    ║
╠═══════════════════╬═══════════╬═══════════╣
║ MLX Active        ║         0 ║     4,358 ║
║ MLX Cache         ║         0 ║     2,473 ║
║ MLX Total         ║         0 ║     6,830 ║
╠═══════════════════╬═══════════╬═══════════╣
║ MLX Peak          ║           ║     5,191 ║
╠═══════════════════╩═══════════╬═══════════╣
║ Time elapsed                  ║     2.6s  ║
╚═══════════════════════════════╩═══════════╝

 Castle Shopping Centre, Banbury, England, United Kingdom, UK, 2025-03-15 16:31:57

Generation time: 2.0s
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^


vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
Processing meta-llama/Llama-3.2-11B-Vision-Instruct
Fetching 15 files: 100%|█████████████████████████████████████████████████████████████| 15/15 [00:00<00:00, 38574.22it/s]
Fetching 15 files: 100%|█████████████████████████████████████████████████████████████| 15/15 [00:00<00:00, 18401.45it/s]
==========
Files: ['/Users/jrp/Pictures/Processed/20250315-163157_DSC02909_openWith-1 copy.jpg'] 

Prompt: <|begin_of_text|><|start_header_id|>user<|end_header_id|>

Provide a factual caption, description and comma-separated keywords or tags for this image so that it can be catalogued and searched for easily. The picture was taken in Castle Shopping Centre, Banbury, England, United Kingdom, UK on 2025-03-15 16:31:57<|image|><|eot_id|><|start_header_id|>assistant<|end_header_id|>



╔═══════════════════╦═══════════╦═══════════╗
║ Memory usage (MB) ║    Pre    ║   Post    ║
╠═══════════════════╬═══════════╬═══════════╣
║ MLX Active        ║         0 ║    22,444 ║
║ MLX Cache         ║       324 ║       772 ║
║ MLX Total         ║       324 ║    23,216 ║
╠═══════════════════╬═══════════╬═══════════╣
║ MLX Peak          ║           ║    22,866 ║
╠═══════════════════╩═══════════╬═══════════╣
║ Time elapsed                  ║     3.5s  ║
╚═══════════════════════════════╩═══════════╝

Model processing error: [metal::Device] Unable to load function steel_attention_bfloat16_bq32_bk16_bd128_wm4_wn1_maskint32
Function steel_attention_bfloat16_bq32_bk16_bd128_wm4_wn1_maskint32 was not found in the library

Traceback (most recent call last):
  File "/Users/jrp/Documents/AI/mlx/scripts/vlm/check_models.py", line 348, in process_model
    output: str = generate(
                  ^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniconda/base/envs/mlx/lib/python3.12/site-packages/mlx_vlm/utils.py", line 1145, in generate
    for response in stream_generate(model, processor, prompt, image, **kwargs):
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniconda/base/envs/mlx/lib/python3.12/site-packages/mlx_vlm/utils.py", line 1066, in stream_generate
    for n, (token, logprobs) in enumerate(
                                ^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniconda/base/envs/mlx/lib/python3.12/site-packages/mlx_vlm/utils.py", line 983, in generate_step
    mx.async_eval(y)
RuntimeError: [metal::Device] Unable to load function steel_attention_bfloat16_bq32_bk16_bd128_wm4_wn1_maskint32
Function steel_attention_bfloat16_bq32_bk16_bd128_wm4_wn1_maskint32 was not found in the library

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^


vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
Processing microsoft/Phi-3.5-vision-instruct
Fetching 14 files: 100%|█████████████████████████████████████████████████████████████| 14/14 [00:00<00:00, 23889.45it/s]
libc++abi: terminating due to uncaught exception of type std::runtime_error: [METAL] Command buffer execution failed: Invalid Resource (00000009:kIOGPUCommandBufferCallbackErrorInvalidResource)
zsh: abort      python check_models.py -v --debug
/opt/homebrew/Caskroom/miniconda/base/envs/mlx/lib/python3.12/multiprocessing/resource_tracker.py:255: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '

[jrp2014]

The script for this result is at https://gist.github.com/jrp2014/4a6e449c11a50caf80924a04dd445109

[Blaizzy]

Can't see the gist

[jrp2014]

Odd. I can't see it either, despite using the sharing link. Anyway:

#!/usr/bin/env python3
"""Image analysis and caption generation using MLX Vision Language Models."""

# Standard library imports
from argparse import ArgumentParser, Namespace
import subprocess
import time
from contextlib import contextmanager
from dataclasses import dataclass
from datetime import datetime
from functools import lru_cache
from pathlib import Path
from typing import (
    Any, Dict, Final, Generator, List, Optional,  NamedTuple,TypedDict
)

# Third-party imports
import mlx.core as mx
from PIL import Image, UnidentifiedImageError
from PIL.ExifTags import GPSTAGS, TAGS

# Local imports
from mlx_vlm import (
    load, generate, __version__ as vlm_version
)
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

# Constants - Defaults
DEFAULT_MAX_TOKENS: Final[int] = 500
DEFAULT_FOLDER: Final[Path] = Path("/Users/jrp/Pictures/Processed")

# Constants - EXIF and IPTC
IMPORTANT_EXIF_TAGS: Final[frozenset[str]] = frozenset({
    "DateTimeOriginal", "ImageDescription", "CreateDate",
    "Make", "Model", "LensModel", "ExposureTime",
    "FNumber", "ISOSpeedRatings", "FocalLength", "ExposureProgram",
})

DATE_FORMATS: Final[tuple[str, ...]] = (
    "%Y:%m:%d %H:%M:%S",
    "%Y-%m-%d %H:%M:%S",
    "%Y%m%d",
)

EXIF_DATE_TAGS: Final[tuple[str, ...]] = (
    "DateTimeOriginal",
    "CreateDate",
    "DateTime"
)

IPTC_TAGS: Final[dict[str, int]] = {
    "LOCATION": 0x5A,
    "DATECREATED": 0x37,
    "CITY": 0x5A,
    "COUNTRY": 0x65,
}

# Type definitions

class MemoryStats(NamedTuple):
    """Memory statistics container."""
    active: float    # Active memory in MB
    cached: float    # Cache memory in MB
    peak: float      # Peak memory in MB
    time: float      # Processing time in seconds

@dataclass(frozen=True)
class ModelResult:
    """Container for model processing results."""
    model_name: str
    output: str
    stats: MemoryStats

class ModelConfig(TypedDict):
    """Type hints for model configuration."""
    chat_template: str
    image_token: str
    eos_token: str

# Memory management
def get_memory_stats() -> Dict[str, float]:
    """Get current memory statistics in MB."""
    return {
        'active': mx.get_active_memory() / 1024 / 1024,
        'cache': mx.get_cache_memory() / 1024 / 1024,
        'peak': mx.get_peak_memory() / 1024 / 1024
    }

@contextmanager
def memory_tracker(verbose: bool = False) -> Generator[Dict[str, float], None, None]:
    """Context manager to track MLX memory usage and timing.
    
    Args:
        verbose: Whether to print memory usage table.
        
    Yields:
        Dict containing active, cached, peak memory and elapsed time.
    """
    start_time: float = time.perf_counter()
    pre_stats: Dict[str, float] = get_memory_stats()
    stats: Dict[str, float] = {'active': 0.0, 'cached': 0.0, 'peak': 0.0, 'time': 0.0}
    
    try:
        yield stats
    finally:
        end_time: float = time.perf_counter()
        post_stats: Dict[str, float] = get_memory_stats()
        total_pre = pre_stats['active'] + pre_stats['cache']
        total_post = post_stats['active'] + post_stats['cache']
        
        stats.update({
            'active': post_stats['active'] - pre_stats['active'],
            'cached': post_stats['cache'] - pre_stats['cache'],
            'peak': post_stats['peak'],
            'time': end_time - start_time,
            'total': total_post - total_pre
        })
        
        if verbose:
            print("\n╔═══════════════════╦═══════════╦═══════════╦═══════════╗")
            print("║ Memory usage (MB) ║    Pre    ║   Post    ║   Delta   ║")
            print("╠═══════════════════╬═══════════╬═══════════╬═══════════╣")
            print(f"║ MLX Active        ║{pre_stats['active']:>10,.0f} ║{post_stats['active']:>10,.0f} ║{stats['active']:>10,.0f} ║")
            print(f"║ MLX Cache         ║{pre_stats['cache']:>10,.0f} ║{post_stats['cache']:>10,.0f} ║{stats['cached']:>10,.0f} ║")
            print(f"║ MLX Total         ║{total_pre:>10,.0f} ║{total_post:>10,.0f} ║{stats['total']:>10,.0f} ║")
            print("╠═══════════════════╬═══════════╬═══════════╬═══════════╣")
            print(f"║ MLX Peak          ║{pre_stats['peak']:>10,.0f} ║{post_stats['peak']:>10,.0f} ║{stats['peak']:>10,.0f} ║")
            print("╠═══════════════════╩═══════════╩═══════════╬═══════════╣")
            print(f"║ Time elapsed                              ║{stats['time']:>8.1f}s  ║")
            print("╚═══════════════════════════════════════════════════════╝\n")

# File handling functions
def get_most_recently_modified_file(folder: str) -> Optional[str]:
    """Return the name of the most recently modified file in the given folder, excluding hidden files."""
    try:
        folder_path: Path = Path(folder)
        if not folder_path.is_dir():
            return None
        files: List[Path] = sorted(
            (f for f in folder_path.iterdir() if not f.name.startswith(".")),
            key=lambda f: f.stat().st_mtime,
            reverse=True,
        )
        return str(files[0]) if files else None
    except Exception as e:
        print(f"Error while getting the most recently modified file: {e}")
        return None


def print_image_dimensions(image_path: str, debug: bool = False) -> None:
    """Print the dimensions of the image given as a path string."""
    try:
        with Image.open(image_path) as img:
            width, height = img.size
            mpx: float = width * height / 1_000_000
            if debug:
                print(f"Image dimensions: {width}x{height} ({mpx:.1f} MPixels)")
    except Exception as e:
        print(f"Error while getting image dimensions: {e}")


def _map_key(k: int) -> str:
    try:
        return TAGS[k]
    except KeyError:
        return GPSTAGS.get(k, k)


@lru_cache(maxsize=128)
def get_exif(image_path: str) -> Optional[Dict[str, Any]]:
    """Get EXIF data from image with caching."""
    try:
        with Image.open(image_path) as img:
            if not hasattr(img, '_getexif') or not (exif := img._getexif()):
                return None
            return {
                TAGS.get(tag_id, tag_id): value
                for tag_id, value in exif.items()
            }
    except (UnidentifiedImageError, OSError) as e:
        print(f"Error reading image file: {e}")
        return None
    except Exception as e:
        print(f"Unexpected error reading EXIF data: {e}")
        return None


def format_date(date_str: str) -> str:
    """Format date string in a consistent way."""
    try:
        for fmt in DATE_FORMATS:
            try:
                return datetime.strptime(date_str, fmt).strftime("%Y-%m-%d %H:%M:%S")
            except ValueError:
                continue
        return date_str
    except Exception:
        return "Unknown date"


def extract_image_metadata(
    image_path: str, 
    verbose: bool = False,
    debug: bool = False,
) -> Dict[str, str]:
    """Extract and format image metadata with improved error handling."""
    metadata: Dict[str, str] = {
        "GPS": "Unknown location",
        "date": "Unknown date",
        "description": "No description",
    }

    try:
        exif = get_exif(image_path)
        if not exif:
            return metadata

        # Extract description
        if desc := exif.get("ImageDescription"):
            metadata["description"] = str(desc).strip(", ")

        # Handle GPS data
        if gps_info := exif.get("GPSInfo"):
            try:
                if debug:
                    print("\nRaw GPS Info:")
                    for k, v in gps_info.items():
                        tag = GPSTAGS.get(k, k)
                        print(f"  {tag}: {v}")

                # Helper function for GPS coordinate conversion
                def get_coordinate(ref_tag: int, coord_tag: int) -> Optional[float]:
                    try:
                        ref = gps_info.get(ref_tag, 'N' if ref_tag == 1 else 'E')
                        if isinstance(ref, bytes):
                            ref = ref.decode()
                        
                        coords = gps_info.get(coord_tag)
                        if not coords or len(coords) != 3:
                            return None

                        degrees = coords[0][0] / coords[0][1] if isinstance(coords[0], tuple) else float(coords[0])
                        minutes = coords[1][0] / coords[1][1] if isinstance(coords[1], tuple) else float(coords[1])
                        seconds = coords[2][0] / coords[2][1] if isinstance(coords[2], tuple) else float(coords[2])

                        decimal = degrees + (minutes / 60.0) + (seconds / 3600.0)
                        return -decimal if ref in ['S', 'W'] else decimal
                    except (IndexError, TypeError, ZeroDivisionError) as e:
                        if debug:
                            print(f"Error converting coordinate: {e}")
                        return None

                # Extract coordinates
                lat = get_coordinate(1, 2)  # 1=GPSLatitudeRef, 2=GPSLatitude
                lon = get_coordinate(3, 4)  # 3=GPSLongitudeRef, 4=GPSLongitude

                if debug:
                    print(f"Extracted coordinates - Latitude: {lat}, Longitude: {lon}")

                if lat is not None and lon is not None:
                    metadata["GPS"] = f"{lat:+.6f}°, {lon:+.6f}°"
                    if debug:
                        print(f"Final GPS string: {metadata['GPS']}")

            except Exception as exc:
                print(f"GPS parsing error: {exc}")
                if debug:
                    import traceback
                    traceback.print_exc()

        # Extract and format date
        for tag in EXIF_DATE_TAGS:
            if date_str := exif.get(tag):
                metadata["date"] = format_date(str(date_str))
                break

        return metadata

    except Exception as exc:
        print(f"Error extracting metadata: {exc}")
        if debug:
            import traceback
            traceback.print_exc()
        return metadata


def pretty_print_exif(exif: Dict[str, Any], verbose: bool = False) -> None:
    """Pretty print EXIF data in a formatted table."""
    if not exif:
        print("No EXIF data found")
        return

    print("\n╔════════════════════════════════╦═══════════════════════════════════════════════╗")
    print("║ EXIF Tag                       ║ Value                                         ║")
    print("╠════════════════════════════════╬═══════════════════════════════════════════════╣")

    for tag, value in sorted(exif.items()):
        # Skip binary data and GPS info (handled separately)
        if isinstance(value, bytes) or tag == "GPSInfo":
            continue

        # Skip if not important and not verbose
        if not verbose and tag not in IMPORTANT_EXIF_TAGS:
            continue

        # Format the tag and value strings
        tag_str = str(tag)[:30]
        value_str = str(value)[:45]

        # Highlight important tags
        if tag in IMPORTANT_EXIF_TAGS:
            print(f"║ \033[1m{tag_str:<30}\033[0m ║ {value_str:<45} ║")
        else:
            print(f"║ {tag_str:<30} ║ {value_str:<45} ║")

    print("╚════════════════════════════════╩═══════════════════════════════════════════════╝")


# Model handling functions
def get_model_paths() -> List[str]:
    """Get list of available models from huggingface cache."""
    try:
        output: bytes = subprocess.check_output(
            [
                "/opt/homebrew/Caskroom/miniconda/base/envs/mlx/bin/huggingface-cli",
                "scan-cache",
            ],
            stderr=subprocess.PIPE,
        )
        return sorted(
            line.split()[0] for line in output.decode("utf-8").split("\n")[2:-4]
        )
    except subprocess.CalledProcessError as e:
        print(f"Error accessing huggingface cache: {e}")
        return []


def print_model_stats(results: List[ModelResult]) -> None:
    """Print a table of model statistics."""
    if not results:
        return

    print("\n╔═══════════════════════════╦═══════════╦═══════════╦═══════════╦═══════════╗")
    print("║ Model                    ║  Active   ║   Cache   ║   Peak    ║   Time    ║")
    print("╠═══════════════════════════╬═══════════╬═══════════╬═══════════╬═══════════╣")
    
    for result in sorted(results, key=lambda x: x.stats.time):
        model_name = result.model_name.split('/')[-1][:20]
        print(
            f"║ {model_name:<21} ║"
            f"{result.stats.active:>9,.0f} ║"
            f"{result.stats.cached:>9,.0f} ║"
            f"{result.stats.peak:>9,.0f} ║"
            f"{result.stats.time:>8.1f}s ║"
        )
    
    if len(results) > 1:
        avg_stats = MemoryStats(
            active=sum(r.stats.active for r in results) / len(results),
            cached=sum(r.stats.cached for r in results) / len(results),
            peak=max(r.stats.peak for r in results),
            time=sum(r.stats.time for r in results) / len(results)
        )
        print("╠═══════════════════════════╬═══════════╬═══════════╬═══════════╬═══════════╣")
        print(
            f"║ AVERAGE                  ║"
            f"{avg_stats.active:>9,.0f} ║"
            f"{avg_stats.cached:>9,.0f} ║"
            f"{avg_stats.peak:>9,.0f} ║"
            f"{avg_stats.time:>8.1f}s ║"
        )
    
    print("╚═══════════════════════════╩═══════════╩═══════════╩═══════════╩═══════════╝")

def process_model(
    model_path: str, 
    image_path: str, 
    prompt: str, 
    verbose: bool, 
    debug: bool = False,
    max_tokens: int = DEFAULT_MAX_TOKENS
) -> Optional[ModelResult]:
    """Process model with MLX memory tracking and error handling."""
    model: Any = None
    tokenizer: Any = None
    
    if model_path == "meta-llama/Llama-3.2-11B-Vision-Instruct":
        return None


    try:
        with memory_tracker(verbose=verbose) as tracking_stats:
            if not Path(image_path).exists():
                raise FileNotFoundError(f"Image file not found: {image_path}")

            trust_remote_code: bool = True
            model, tokenizer = load(model_path, trust_remote_code=trust_remote_code)
            config: ModelConfig = load_config(model_path, trust_remote_code=trust_remote_code)

            formatted_prompt: str = apply_chat_template(
                tokenizer, config, prompt, num_images=1
            )

            start_time: float = time.perf_counter()
            output: str = generate(
                model, tokenizer, formatted_prompt, [image_path], 
                verbose=verbose, max_tokens=max_tokens
            )
            time_taken: float = time.perf_counter() - start_time

            # Include total memory in stats
            return ModelResult(
                model_name=model_path,
                output=output,
                stats=MemoryStats(
                    active=tracking_stats['active'],
                    cached=tracking_stats['cached'],
                    peak=tracking_stats['peak'],
                    time=time_taken
                )
            )
    except FileNotFoundError as e:
        print(f"File not found: {e}")
    except Exception as exc:
        print(f"Model error ({type(exc).__name__}): {exc}")
        if debug:
            import traceback
            traceback.print_exc()
    finally:
        if model is not None:
            del model
        if tokenizer is not None:
            del tokenizer
        mx.metal.clear_cache()
        mx.metal.reset_peak_memory()
    
    return None

def main(
    folder: str,
    prompt: Optional[str] = None,
    verbose: bool = False,
    max_tokens: int = DEFAULT_MAX_TOKENS,
    debug: bool = False,
) -> None:
    """Process images with MLX vision language models."""
    start_time: float = time.perf_counter()
    results: List[ModelResult] = []

    if not Path(folder).is_dir():
        print(f"Error: {folder} is not a valid directory")
        return

    if most_recent_file := get_most_recently_modified_file(folder):
        print(f"\nProcessing file: {most_recent_file}")
        print_image_dimensions(most_recent_file, debug=debug)

        if metadata := extract_image_metadata(most_recent_file, verbose, debug):
            if debug:
                print(
                    f"Lat & Longitude: {metadata['GPS']}\n"
                    f"Date:            {metadata['date']}\n"
                    f"Description:     {metadata['description']}"
                )

            default_prompt: str = (
                "Provide a factual caption, description and comma-separated "
                "keywords or tags for this image so that it can be catalogued "
                "and searched for easily. The picture was taken in {metadata['description']} "
                "on {metadata['date']}"
            )
            if metadata['GPS'] != "Unknown location":
                default_prompt += f" from the GPS location {metadata['GPS']}"

            actual_prompt: str = default_prompt if prompt is None else prompt

            models: List[str] = get_model_paths()
            if not models:
                print("No models found in huggingface cache")
                return

            for model_path in models:
                print(f"\n{80 * 'v'}")
                print(f"\033[1mProcessing {model_path}\033[0m")

                if result := process_model(
                    model_path, most_recent_file, actual_prompt, verbose, debug, max_tokens
                ):
                    # Print model output and stats only if verbose is not set, as it will already have been printed by generate otherwise
                    if not verbose:
                        print(result.output)
                    print(f"\nGeneration time: {result.stats.time:.1f}s")
                    results.append(result)

                print(f"{80 * '^'}\n")

            if results:
                print_model_stats(results)
    else:
        print("No files found in the folder.")

    elapsed: float = time.perf_counter() - start_time
    print(f"\nTotal execution time: {elapsed:.1f}s")


if __name__ == "__main__":
    print(f"\nMLX version: {mx.__version__}")
    print(f"MLX-VLM version: {vlm_version}\n")

    parser: ArgumentParser = ArgumentParser(
        description="Describe, caption and keyword the most recently modified image in a folder"
    )
    parser.add_argument(
        "-f",
        "--folder",
        type=str,
        default=DEFAULT_FOLDER,
        help="The folder to scan for the most recently modified file",
    )
    parser.add_argument(
        "-v",
        "--verbose",
        action="store_true",
        help="Enable verbose output",
    )
    parser.add_argument(
        "-p",
        "--prompt",
        type=str,
        help="Custom prompt to use for image analysis",
    )
    parser.add_argument(
        "-m",
        "--max-tokens",
        type=int,
        default=DEFAULT_MAX_TOKENS,
        help="Maximum number of tokens to generate (default: 500)",
    )
    parser.add_argument(
        "-d",
        "--debug",
        action="store_true",
        help="Enable debug output",
    )

    args: Namespace = parser.parse_args()
    main(args.folder, args.prompt, args.verbose, args.max_tokens, args.debug)

[jrp2014]

It seems to be working again... I suspect that it was an update to transformers or mlx or some such. Some of the models don't run. It would be good if the generate and other key api functions also returned performance stats.

python check_models.py --verbose        

MLX version: 0.24.1.dev20250328+05d71185
MLX-VLM version: 0.1.21


Processing file: /Users/jrp/Pictures/Processed/20250329-163408_DSC03146.jpg

vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
Processing HuggingFaceTB/SmolVLM-Instruct
Fetching 12 files: 100%|█████████████████████████████████████████████████████████████| 12/12 [00:00<00:00, 17142.93it/s]
Fetching 12 files: 100%|█████████████████████████████████████████████████████████████| 12/12 [00:00<00:00, 34592.20it/s]
==========
Files: ['/Users/jrp/Pictures/Processed/20250329-163408_DSC03146.jpg'] 

Prompt: <|im_start|>User:<image>Provide a factual caption, description and comma-separated keywords or tags for this image so that it can be catalogued and searched for easily. The picture was taken in River Adur, Shoreham-by-Sea, England, United Kingdom, UK on 2025-03-29 16:34:08 from the GPS location +50.829800°, -0.272917°. Do not include this GPS location or the date in your response.<end_of_utterance>
Assistant:
mx.metal.get_peak_memory is deprecated and will be removed in a future version. Use mx.get_peak_memory instead.
 Two old sailboats sit in the mud next to each other in a harbor. The boats are both white and blue. The one on the left is smaller and has a mast while the one on the right is larger and has no mast. There are many other sailboats in the harbor. There is a building in the background.
==========
Prompt: 1303 tokens, 1102.260 tokens-per-sec
Generation: 67 tokens, 118.087 tokens-per-sec
Peak memory: 5.443 GB


Model output:

 Two old sailboats sit in the mud next to each other in a harbor. The boats are both white and blue. The one on the left is smaller and has a mast while the one on the right is larger and has no mast. There are many other sailboats in the harbor. There is a building in the background.

Generation time: 3.2s

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^


vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
Processing meta-llama/Llama-3.2-11B-Vision-Instruct
Fetching 15 files: 100%|█████████████████████████████████████████████████████████████| 15/15 [00:00<00:00, 20216.76it/s]
Fetching 15 files: 100%|█████████████████████████████████████████████████████████████| 15/15 [00:00<00:00, 33770.56it/s]
==========
Files: ['/Users/jrp/Pictures/Processed/20250329-163408_DSC03146.jpg'] 

Prompt: <|begin_of_text|><|start_header_id|>user<|end_header_id|>

Provide a factual caption, description and comma-separated keywords or tags for this image so that it can be catalogued and searched for easily. The picture was taken in River Adur, Shoreham-by-Sea, England, United Kingdom, UK on 2025-03-29 16:34:08 from the GPS location +50.829800°, -0.272917°. Do not include this GPS location or the date in your response.<|image|><|eot_id|><|start_header_id|>assistant<|end_header_id|>


The image depicts a serene scene of boats beached on the shore of a river, with a small town in the background. The photograph was taken in the UK.

**Caption:** "Boats on the shore of a river in a small town in the UK."

**Description:** The image shows a group of boats, some of which are old and weathered, beached on the shore of a river. The boats are in various states of disrepair, with some appearing to be abandoned or neglected. The shore is covered in mud and grass, and the sky is overcast.

**Keywords:** boats, river, shore, small town, UK, abandoned, neglected, weathered, overcast, sky, mud, grass.
==========
Prompt: 105 tokens, 25.839 tokens-per-sec
Generation: 148 tokens, 3.777 tokens-per-sec
Peak memory: 25.023 GB


Model output:

The image depicts a serene scene of boats beached on the shore of a river, with a small town in the background. The photograph was taken in the UK.

**Caption:** "Boats on the shore of a river in a small town in the UK."

**Description:** The image shows a group of boats, some of which are old and weathered, beached on the shore of a river. The boats are in various states of disrepair, with some appearing to be abandoned or neglected. The shore is covered in mud and grass, and the sky is overcast.

**Keywords:** boats, river, shore, small town, UK, abandoned, neglected, weathered, overcast, sky, mud, grass.

Generation time: 46.4s

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^


vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
Processing microsoft/Florence-2-large-ft
Fetching 9 files: 100%|█████████████████████████████████████████████████████████████████| 9/9 [00:00<00:00, 6293.55it/s]
ERROR:root:No safetensors found in /Users/jrp/.cache/huggingface/hub/models--microsoft--Florence-2-large-ft/snapshots/bb44b80c15e943b1bf7cec6e076359cec6e40178
File not found: 
No safetensors found in /Users/jrp/.cache/huggingface/hub/models--microsoft--Florence-2-large-ft/snapshots/bb44b80c15e943b1bf7cec6e076359cec6e40178
Create safetensors using the following code:

from transformers import AutoModelForCausalLM, AutoProcessor

model_id= "<huggingface_model_id>"
model = AutoModelForCausalLM.from_pretrained(model_id)
processor = AutoProcessor.from_pretrained(model_id)

model.save_pretrained("<local_dir>")
processor.save_pretrained("<local_dir>")

Then use the <local_dir> as the --hf-path in the convert script.

python -m mlx_vlm.convert --hf-path <local_dir> --mlx-path <mlx_dir>

        
Error processing model: microsoft/Florence-2-large-ft
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^


vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
Processing microsoft/Phi-3.5-vision-instruct
Fetching 14 files: 100%|█████████████████████████████████████████████████████████████| 14/14 [00:00<00:00, 11903.56it/s]
/opt/homebrew/Caskroom/miniconda/base/envs/mlx/lib/python3.12/site-packages/transformers/models/auto/image_processing_auto.py:602: FutureWarning: The image_processor_class argument is deprecated and will be removed in v4.42. Please use `slow_image_processor_class`, or `fast_image_processor_class` instead
  warnings.warn(
Fetching 14 files: 100%|█████████████████████████████████████████████████████████████| 14/14 [00:00<00:00, 15658.73it/s]
==========
Files: ['/Users/jrp/Pictures/Processed/20250329-163408_DSC03146.jpg'] 

Prompt: <|user|>
<|image_1|>Provide a factual caption, description and comma-separated keywords or tags for this image so that it can be catalogued and searched for easily. The picture was taken in River Adur, Shoreham-by-Sea, England, United Kingdom, UK on 2025-03-29 16:34:08 from the GPS location +50.829800°, -0.272917°. Do not include this GPS location or the date in your response.<|end|>
<|assistant|>

The image shows a serene waterfront scene with several boats moored in a calm river. The boats vary in size and design, with some appearing to be traditional sailing vessels and others that seem to be motorboats. The water is calm, reflecting the overcast sky above. In the background, there are buildings that suggest this might be a small town or a residential area. The overall atmosphere of the image is peaceful and quiet, with no visible activity or movement.<|end|>
==========
Prompt: 885 tokens, 884.967 tokens-per-sec
Generation: 101 tokens, 10.173 tokens-per-sec
Peak memory: 10.941 GB


Model output:

The image shows a serene waterfront scene with several boats moored in a calm river. The boats vary in size and design, with some appearing to be traditional sailing vessels and others that seem to be motorboats. The water is calm, reflecting the overcast sky above. In the background, there are buildings that suggest this might be a small town or a residential area. The overall atmosphere of the image is peaceful and quiet, with no visible activity or movement.<|end|>

Generation time: 13.2s

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^


vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
Processing mlx-community/Florence-2-large-ft-bf16
Fetching 12 files: 100%|█████████████████████████████████████████████████████████████| 12/12 [00:00<00:00, 14405.17it/s]
Fetching 12 files: 100%|█████████████████████████████████████████████████████████████| 12/12 [00:00<00:00, 15270.52it/s]
==========
Files: ['/Users/jrp/Pictures/Processed/20250329-163408_DSC03146.jpg'] 

Prompt: Provide a factual caption, description and comma-separated keywords or tags for this image so that it can be catalogued and searched for easily. The picture was taken in River Adur, Shoreham-by-Sea, England, United Kingdom, UK on 2025-03-29 16:34:08 from the GPS location +50.829800°, -0.272917°. Do not include this GPS location or the date in your response.
<s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s>
==========
Prompt: 98 tokens, 330.320 tokens-per-sec
Generation: 500 tokens, 177.328 tokens-per-sec
Peak memory: 3.457 GB


Model output:

<s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s><s>

Generation time: 4.8s

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^


vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
Processing mlx-community/Idefics3-8B-Llama3-bf16
Fetching 12 files: 100%|█████████████████████████████████████████████████████████████| 12/12 [00:00<00:00, 13425.35it/s]
Fetching 12 files: 100%|█████████████████████████████████████████████████████████████| 12/12 [00:00<00:00, 36604.83it/s]
==========
Files: ['/Users/jrp/Pictures/Processed/20250329-163408_DSC03146.jpg'] 

Prompt: <|begin_of_text|>User:<image>Provide a factual caption, description and comma-separated keywords or tags for this image so that it can be catalogued and searched for easily. The picture was taken in River Adur, Shoreham-by-Sea, England, United Kingdom, UK on 2025-03-29 16:34:08 from the GPS location +50.829800°, -0.272917°. Do not include this GPS location or the date in your response.<end_of_utterance>
Assistant:
 Two boats are beached in the mud in the town of Shoreham-by-Sea.<end_of_utterance>
==========
Prompt: 2416 tokens, 647.824 tokens-per-sec
Generation: 20 tokens, 31.011 tokens-per-sec
Peak memory: 18.436 GB


Model output:

 Two boats are beached in the mud in the town of Shoreham-by-Sea.<end_of_utterance>

Generation time: 8.8s

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^


vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
Processing mlx-community/Llama-3.2-11B-Vision-Instruct-8bit
Fetching 10 files: 100%|█████████████████████████████████████████████████████████████| 10/10 [00:00<00:00, 50533.78it/s]
Fetching 10 files: 100%|█████████████████████████████████████████████████████████████| 10/10 [00:00<00:00, 25312.64it/s]
==========
Files: ['/Users/jrp/Pictures/Processed/20250329-163408_DSC03146.jpg'] 

Prompt: <|begin_of_text|><|start_header_id|>user<|end_header_id|>

Provide a factual caption, description and comma-separated keywords or tags for this image so that it can be catalogued and searched for easily. The picture was taken in River Adur, Shoreham-by-Sea, England, United Kingdom, UK on 2025-03-29 16:34:08 from the GPS location +50.829800°, -0.272917°. Do not include this GPS location or the date in your response.<|image|><|eot_id|><|start_header_id|>assistant<|end_header_id|>


The image depicts a serene scene of boats in a marshy area, with a small town in the background. The boats are old and weathered, with peeling paint and rusty hulls, but they still exude a sense of charm and character.

* Boats:
	+ There are several boats in the image, all of which are old and weathered.
	+ The boats are moored in a marshy area, with some of them partially submerged in the water.
	+ The boats have peeling paint and rusty hulls, but they still look like they could be used for sailing or other water activities.
* Water:
	+ The water is murky and brown, with a lot of mud and debris in it.
	+ The water is shallow, with some areas of the marshy ground visible above the surface.
	+ The water is calm, with no signs of waves or ripples.
* Land:
	+ The land is flat and marshy, with a lot of mud and water in it.
	+ The land is covered in grass and other vegetation, with some areas of bare ground visible.
	+ The land is surrounded by water, with no signs of any other landmasses nearby.
* Sky:
	+ The sky is cloudy and overcast, with a lot of grey and white clouds.
	+ The sky is not very bright, with the sun hidden behind the clouds.
	+ The sky is calm, with no signs of any storms or other weather-related events.

Overall, the image presents a peaceful and serene scene of boats in a marshy area, with a small town in the background. The image is well-composed, with a good balance of elements and a clear focus on the boats and the surrounding environment.
==========
Prompt: 104 tokens, 39.912 tokens-per-sec
Generation: 357 tokens, 8.826 tokens-per-sec
Peak memory: 14.965 GB


Model output:

The image depicts a serene scene of boats in a marshy area, with a small town in the background. The boats are old and weathered, with peeling paint and rusty hulls, but they still exude a sense of charm and character.

* Boats:
	+ There are several boats in the image, all of which are old and weathered.
	+ The boats are moored in a marshy area, with some of them partially submerged in the water.
	+ The boats have peeling paint and rusty hulls, but they still look like they could be used for sailing or other water activities.
* Water:
	+ The water is murky and brown, with a lot of mud and debris in it.
	+ The water is shallow, with some areas of the marshy ground visible above the surface.
	+ The water is calm, with no signs of waves or ripples.
* Land:
	+ The land is flat and marshy, with a lot of mud and water in it.
	+ The land is covered in grass and other vegetation, with some areas of bare ground visible.
	+ The land is surrounded by water, with no signs of any other landmasses nearby.
* Sky:
	+ The sky is cloudy and overcast, with a lot of grey and white clouds.
	+ The sky is not very bright, with the sun hidden behind the clouds.
	+ The sky is calm, with no signs of any storms or other weather-related events.

Overall, the image presents a peaceful and serene scene of boats in a marshy area, with a small town in the background. The image is well-composed, with a good balance of elements and a clear focus on the boats and the surrounding environment.

Generation time: 46.4s

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^


vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
Processing mlx-community/Llama-3.2-90B-Vision-Instruct-4bit
Fetching 17 files: 100%|██████████████████████████████████████████████████████████████| 17/17 [00:00<00:00, 9652.52it/s]
Fetching 17 files: 100%|█████████████████████████████████████████████████████████████| 17/17 [00:00<00:00, 15453.66it/s]
==========
Files: ['/Users/jrp/Pictures/Processed/20250329-163408_DSC03146.jpg'] 

Prompt: <|begin_of_text|><|start_header_id|>user<|end_header_id|>

Provide a factual caption, description and comma-separated keywords or tags for this image so that it can be catalogued and searched for easily. The picture was taken in River Adur, Shoreham-by-Sea, England, United Kingdom, UK on 2025-03-29 16:34:08 from the GPS location +50.829800°, -0.272917°. Do not include this GPS location or the date in your response.<|image|><|eot_id|><|start_header_id|>assistant<|end_header_id|>


**Caption:** Two old boats on the shore of River Adur in Shoreham-by-Sea, England, UK.

**Description:** This image shows two old boats on the shore of River Adur in Shoreham-by-Sea, England, UK. The boats are white and blue and appear to be in a state of disrepair. They are sitting on the shore, surrounded by grass and mud. In the background, there are several other boats and buildings, including a marina and a church tower.

**Keywords:** River Adur, Shoreham-by-Sea, England, UK, boats, marina, church, tower, old, disrepair, grass, mud, water, sky, clouds, day, daytime, outdoor, outdoors, nature, landscape, travel, tourism, history, culture, architecture, building, structure, transportation, transport, vehicle, watercraft, leisure, recreation, sport, activity, adventure, exploration, discovery, experience, enjoyment, pleasure, relaxation, rest, relaxation, holiday, vacation, weekend, getaway, escape, adventure, exploration, discovery, experience, enjoyment, pleasure, relaxation, rest, relaxation, holiday, vacation, weekend, getaway, escape, adventure, exploration, discovery, experience, enjoyment, pleasure, relaxation, rest, relaxation, holiday, vacation, weekend, getaway, escape, adventure, exploration, discovery, experience, enjoyment, pleasure, relaxation, rest, relaxation, holiday, vacation, weekend, getaway, escape, adventure, exploration, discovery, experience, enjoyment, pleasure, relaxation, rest, relaxation, holiday, vacation, weekend, getaway, escape, adventure, exploration, discovery, experience, enjoyment, pleasure, relaxation, rest, relaxation, holiday, vacation, weekend, getaway, escape, adventure, exploration, discovery, experience, enjoyment, pleasure, relaxation, rest, relaxation, holiday, vacation, weekend, getaway, escape, adventure, exploration, discovery, experience, enjoyment, pleasure, relaxation, rest, relaxation, holiday, vacation, weekend, getaway, escape, adventure, exploration, discovery, experience, enjoyment, pleasure, relaxation, rest, relaxation, holiday, vacation, weekend, getaway, escape, adventure, exploration, discovery, experience, enjoyment, pleasure, relaxation, rest, relaxation, holiday, vacation, weekend, getaway, escape, adventure, exploration, discovery, experience, enjoyment, pleasure, relaxation, rest, relaxation, holiday, vacation, weekend, getaway, escape, adventure, exploration, discovery,
==========
Prompt: 105 tokens, 11.416 tokens-per-sec
Generation: 500 tokens, 1.682 tokens-per-sec
Peak memory: 53.509 GB


Model output:

**Caption:** Two old boats on the shore of River Adur in Shoreham-by-Sea, England, UK.

**Description:** This image shows two old boats on the shore of River Adur in Shoreham-by-Sea, England, UK. The boats are white and blue and appear to be in a state of disrepair. They are sitting on the shore, surrounded by grass and mud. In the background, there are several other boats and buildings, including a marina and a church tower.

**Keywords:** River Adur, Shoreham-by-Sea, England, UK, boats, marina, church, tower, old, disrepair, grass, mud, water, sky, clouds, day, daytime, outdoor, outdoors, nature, landscape, travel, tourism, history, culture, architecture, building, structure, transportation, transport, vehicle, watercraft, leisure, recreation, sport, activity, adventure, exploration, discovery, experience, enjoyment, pleasure, relaxation, rest, relaxation, holiday, vacation, weekend, getaway, escape, adventure, exploration, discovery, experience, enjoyment, pleasure, relaxation, rest, relaxation, holiday, vacation, weekend, getaway, escape, adventure, exploration, discovery, experience, enjoyment, pleasure, relaxation, rest, relaxation, holiday, vacation, weekend, getaway, escape, adventure, exploration, discovery, experience, enjoyment, pleasure, relaxation, rest, relaxation, holiday, vacation, weekend, getaway, escape, adventure, exploration, discovery, experience, enjoyment, pleasure, relaxation, rest, relaxation, holiday, vacation, weekend, getaway, escape, adventure, exploration, discovery, experience, enjoyment, pleasure, relaxation, rest, relaxation, holiday, vacation, weekend, getaway, escape, adventure, exploration, discovery, experience, enjoyment, pleasure, relaxation, rest, relaxation, holiday, vacation, weekend, getaway, escape, adventure, exploration, discovery, experience, enjoyment, pleasure, relaxation, rest, relaxation, holiday, vacation, weekend, getaway, escape, adventure, exploration, discovery, experience, enjoyment, pleasure, relaxation, rest, relaxation, holiday, vacation, weekend, getaway, escape, adventure, exploration, discovery, experience, enjoyment, pleasure, relaxation, rest, relaxation, holiday, vacation, weekend, getaway, escape, adventure, exploration, discovery, experience, enjoyment, pleasure, relaxation, rest, relaxation, holiday, vacation, weekend, getaway, escape, adventure, exploration, discovery,

Generation time: 319.0s

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^


vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
Processing mlx-community/Molmo-7B-D-0924-8bit
Fetching 16 files: 100%|█████████████████████████████████████████████████████████████| 16/16 [00:00<00:00, 14788.20it/s]
Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.52, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`.
Fetching 16 files: 100%|██████████████████████████████████████████████████████████████| 16/16 [00:00<00:00, 8135.39it/s]
==========
Files: ['/Users/jrp/Pictures/Processed/20250329-163408_DSC03146.jpg'] 

Prompt: Provide a factual caption, description and comma-separated keywords or tags for this image so that it can be catalogued and searched for easily. The picture was taken in River Adur, Shoreham-by-Sea, England, United Kingdom, UK on 2025-03-29 16:34:08 from the GPS location +50.829800°, -0.272917°. Do not include this GPS location or the date in your response.
 Sailboats, beached, marshland, water, sky, horizon, town, buildings, shore, England, United Kingdom, UK, River Adur, Shoreham-by-Sea, England, United Kingdom, UK, March 29, 2025, afternoon, daylight, clear, calm, serene, abandoned, rustic, natural, coastal, maritime, nautical, boating, storage, maintenance, weathered, aged, industrial, urban, landscape, shoreline, waterway, river, estuary, marina, harbor, dock, jetty, pier, wharf, town, city, townships, buildings, structures, architecture, townships, landscape, environment, nature, ecosystem, wetlands, marshes, waterways, coastal regions, shorelines, maritime activities, boating infrastructure, vessel storage, seasonal changes, weather conditions, environmental factors, human interaction with nature, coastal development, town planning, urban-rural interface, economic activities, leisure industries, transportation systems, historical significance, cultural heritage, environmental management, land use, resource utilization, sustainable development, coastal management, marine conservation, local economy, tourism, recreation, water sports, boating culture, maritime traditions, town life, community activities, seasonal variations, weather patterns, environmental monitoring, coastal protection, land use changes, human impact on nature, town growth, urban expansion, coastal defense, flood risk management, environmental impact assessment, coastal zone management, land use planning, town development strategies, local government responsibilities, coastal management boards, environmental protection agencies, town planning commissions, marine resource management, coastal zone regulations, land use zonings, coastal zone management plans, local environmental policies, coastal zone development guidelines, land use change monitoring, coastal zone assessment, land use statistics, coastal zone inventory, land use classification systems, coastal zone management best practices, land use planning software, coastal zone management tools, land use change detection techniques, coastal zone management frameworks, land use planning software, coastal zone assessment methodologies, land use change detection software, coastal zone management strategies, land use planning principles, coastal zone assessment tools, land use change detection software, coastal zone management frameworks, land use planning principles, coastal zone assessment tools, land use change detection software, coastal zone management frameworks, land use planning principles, coastal zone assessment tools, land use change detection software, coastal zone management frameworks, land use planning principles, coastal zone assessment tools, land use change detection software, coastal zone management frameworks,
==========
Prompt: 1318 tokens, 37.111 tokens-per-sec
Generation: 500 tokens, 41.938 tokens-per-sec
Peak memory: 41.652 GB


Model output:

 Sailboats, beached, marshland, water, sky, horizon, town, buildings, shore, England, United Kingdom, UK, River Adur, Shoreham-by-Sea, England, United Kingdom, UK, March 29, 2025, afternoon, daylight, clear, calm, serene, abandoned, rustic, natural, coastal, maritime, nautical, boating, storage, maintenance, weathered, aged, industrial, urban, landscape, shoreline, waterway, river, estuary, marina, harbor, dock, jetty, pier, wharf, town, city, townships, buildings, structures, architecture, townships, landscape, environment, nature, ecosystem, wetlands, marshes, waterways, coastal regions, shorelines, maritime activities, boating infrastructure, vessel storage, seasonal changes, weather conditions, environmental factors, human interaction with nature, coastal development, town planning, urban-rural interface, economic activities, leisure industries, transportation systems, historical significance, cultural heritage, environmental management, land use, resource utilization, sustainable development, coastal management, marine conservation, local economy, tourism, recreation, water sports, boating culture, maritime traditions, town life, community activities, seasonal variations, weather patterns, environmental monitoring, coastal protection, land use changes, human impact on nature, town growth, urban expansion, coastal defense, flood risk management, environmental impact assessment, coastal zone management, land use planning, town development strategies, local government responsibilities, coastal management boards, environmental protection agencies, town planning commissions, marine resource management, coastal zone regulations, land use zonings, coastal zone management plans, local environmental policies, coastal zone development guidelines, land use change monitoring, coastal zone assessment, land use statistics, coastal zone inventory, land use classification systems, coastal zone management best practices, land use planning software, coastal zone management tools, land use change detection techniques, coastal zone management frameworks, land use planning software, coastal zone assessment methodologies, land use change detection software, coastal zone management strategies, land use planning principles, coastal zone assessment tools, land use change detection software, coastal zone management frameworks, land use planning principles, coastal zone assessment tools, land use change detection software, coastal zone management frameworks, land use planning principles, coastal zone assessment tools, land use change detection software, coastal zone management frameworks, land use planning principles, coastal zone assessment tools, land use change detection software, coastal zone management frameworks,

Generation time: 50.4s

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^


vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
Processing mlx-community/Molmo-7B-D-0924-bf16
Fetching 18 files: 100%|█████████████████████████████████████████████████████████████| 18/18 [00:00<00:00, 10766.90it/s]
Fetching 18 files: 100%|█████████████████████████████████████████████████████████████| 18/18 [00:00<00:00, 19050.59it/s]
==========
Files: ['/Users/jrp/Pictures/Processed/20250329-163408_DSC03146.jpg'] 

Prompt: Provide a factual caption, description and comma-separated keywords or tags for this image so that it can be catalogued and searched for easily. The picture was taken in River Adur, Shoreham-by-Sea, England, United Kingdom, UK on 2025-03-29 16:34:08 from the GPS location +50.829800°, -0.272917°. Do not include this GPS location or the date in your response.
 Sailboats, beached, marshland, water, sky, horizon, town, buildings, shore, England, United Kingdom, UK, River Adur, March, afternoon, lighting, composition, color, landscape, maritime, boats, shorelines, natural setting, coastal, environment, weather, seasonal, atmospheric, lighting conditions, photographic style, realistic, representational, outdoor, recreational, boating, marine, coastal, English, landscape photography, sailboat photography, beach photography, marshland photography
==========
Prompt: 1318 tokens, 36.824 tokens-per-sec
Generation: 102 tokens, 27.217 tokens-per-sec
Peak memory: 48.575 GB


Model output:

 Sailboats, beached, marshland, water, sky, horizon, town, buildings, shore, England, United Kingdom, UK, River Adur, March, afternoon, lighting, composition, color, landscape, maritime, boats, shorelines, natural setting, coastal, environment, weather, seasonal, atmospheric, lighting conditions, photographic style, realistic, representational, outdoor, recreational, boating, marine, coastal, English, landscape photography, sailboat photography, beach photography, marshland photography

Generation time: 43.1s

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^


vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
Processing mlx-community/Phi-3.5-vision-instruct-bf16
Fetching 13 files: 100%|█████████████████████████████████████████████████████████████| 13/13 [00:00<00:00, 14820.86it/s]
Fetching 13 files: 100%|█████████████████████████████████████████████████████████████| 13/13 [00:00<00:00, 15402.81it/s]
==========
Files: ['/Users/jrp/Pictures/Processed/20250329-163408_DSC03146.jpg'] 

Prompt: <|user|>
<|image_1|>Provide a factual caption, description and comma-separated keywords or tags for this image so that it can be catalogued and searched for easily. The picture was taken in River Adur, Shoreham-by-Sea, England, United Kingdom, UK on 2025-03-29 16:34:08 from the GPS location +50.829800°, -0.272917°. Do not include this GPS location or the date in your response.<|end|>
<|assistant|>

The image shows a serene waterfront scene with several boats moored in a calm river. The boats vary in size and design, with some appearing to be traditional sailing vessels and others that seem to be motorboats. The water is calm, reflecting the overcast sky above. In the background, there are buildings that suggest this might be a small town or a residential area. The overall atmosphere of the image is peaceful and quiet, with no visible activity or movement.<|end|>
==========
Prompt: 885 tokens, 893.011 tokens-per-sec
Generation: 101 tokens, 10.154 tokens-per-sec
Peak memory: 10.941 GB


Model output:

The image shows a serene waterfront scene with several boats moored in a calm river. The boats vary in size and design, with some appearing to be traditional sailing vessels and others that seem to be motorboats. The water is calm, reflecting the overcast sky above. In the background, there are buildings that suggest this might be a small town or a residential area. The overall atmosphere of the image is peaceful and quiet, with no visible activity or movement.<|end|>

Generation time: 13.1s

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^


vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
Processing mlx-community/QwQ-32B-4bit
Fetching 12 files: 100%|█████████████████████████████████████████████████████████████| 12/12 [00:00<00:00, 24338.32it/s]
ERROR:root:Model type qwen2 not supported.
Model error (ValueError): Model type qwen2 not supported.
Error processing model: mlx-community/QwQ-32B-4bit
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^


vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
Processing mlx-community/QwQ-32B-8bit
Fetching 15 files: 100%|██████████████████████████████████████████████████████████████| 15/15 [00:00<00:00, 9231.78it/s]
ERROR:root:Model type qwen2 not supported.
Model error (ValueError): Model type qwen2 not supported.
Error processing model: mlx-community/QwQ-32B-8bit
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^


vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
Processing mlx-community/SmolVLM-Instruct-bf16
Fetching 12 files: 100%|█████████████████████████████████████████████████████████████| 12/12 [00:00<00:00, 59423.43it/s]
Fetching 12 files: 100%|█████████████████████████████████████████████████████████████| 12/12 [00:00<00:00, 12658.87it/s]
==========
Files: ['/Users/jrp/Pictures/Processed/20250329-163408_DSC03146.jpg'] 

Prompt: <|im_start|>User:<image>Provide a factual caption, description and comma-separated keywords or tags for this image so that it can be catalogued and searched for easily. The picture was taken in River Adur, Shoreham-by-Sea, England, United Kingdom, UK on 2025-03-29 16:34:08 from the GPS location +50.829800°, -0.272917°. Do not include this GPS location or the date in your response.<end_of_utterance>
Assistant:
 Two old sailboats sit in the mud next to each other in a harbor. The boats are both white and blue. The one on the left is smaller and has a mast while the one on the right is larger and has no mast. There are many other sailboats in the harbor. There is a building in the background.
==========
Prompt: 1303 tokens, 1101.665 tokens-per-sec
Generation: 67 tokens, 117.864 tokens-per-sec
Peak memory: 5.443 GB


Model output:

 Two old sailboats sit in the mud next to each other in a harbor. The boats are both white and blue. The one on the left is smaller and has a mast while the one on the right is larger and has no mast. There are many other sailboats in the harbor. There is a building in the background.

Generation time: 3.5s

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^


vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
Processing mlx-community/SmolVLM2-2.2B-Instruct-mlx
Fetching 12 files: 100%|█████████████████████████████████████████████████████████████| 12/12 [00:00<00:00, 10837.99it/s]
Fetching 12 files: 100%|█████████████████████████████████████████████████████████████| 12/12 [00:00<00:00, 15210.53it/s]
==========
Files: ['/Users/jrp/Pictures/Processed/20250329-163408_DSC03146.jpg'] 

Prompt: <|im_start|>User:<image>Provide a factual caption, description and comma-separated keywords or tags for this image so that it can be catalogued and searched for easily. The picture was taken in River Adur, Shoreham-by-Sea, England, United Kingdom, UK on 2025-03-29 16:34:08 from the GPS location +50.829800°, -0.272917°. Do not include this GPS location or the date in your response.<end_of_utterance>
Assistant:
 Two old boats sit on the shore of a muddy river.
==========
Prompt: 1203 tokens, 1044.781 tokens-per-sec
Generation: 13 tokens, 125.977 tokens-per-sec
Peak memory: 5.444 GB


Model output:

 Two old boats sit on the shore of a muddy river.

Generation time: 3.0s

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^


vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
Processing mlx-community/deepseek-vl2-8bit
Fetching 13 files: 100%|█████████████████████████████████████████████████████████████| 13/13 [00:00<00:00, 66252.68it/s]
Some kwargs in processor config are unused and will not have any effect: candidate_resolutions, normalize, pad_token, image_mean, downsample_ratio, sft_format, mask_prompt, add_special_token, image_token, image_std, ignore_id, patch_size, chat_template. 
Add pad token = ['<｜▁pad▁｜>'] to the tokenizer
<｜▁pad▁｜>:2
Add image token = ['<image>'] to the tokenizer
<image>:128815
Added grounding-related tokens
Added chat tokens
Fetching 13 files: 100%|█████████████████████████████████████████████████████████████| 13/13 [00:00<00:00, 10235.77it/s]
==========
Files: ['/Users/jrp/Pictures/Processed/20250329-163408_DSC03146.jpg'] 

Prompt: <|User|>: <image>
Provide a factual caption, description and comma-separated keywords or tags for this image so that it can be catalogued and searched for easily. The picture was taken in River Adur, Shoreham-by-Sea, England, United Kingdom, UK on 2025-03-29 16:34:08 from the GPS location +50.829800°, -0.272917°. Do not include this GPS location or the date in your response.

<|Assistant|>:
Sandy shores with moored sailboats, water, and a distant view of a small town with a mix of modern and traditional buildings.
==========
Prompt: 1529 tokens, 204.024 tokens-per-sec
Generation: 29 tokens, 66.911 tokens-per-sec
Peak memory: 31.235 GB


Model output:

Sandy shores with moored sailboats, water, and a distant view of a small town with a mix of modern and traditional buildings.

Generation time: 13.4s

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^


vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
Processing mlx-community/gemma-3-12b-pt-8bit
Fetching 13 files: 100%|█████████████████████████████████████████████████████████████| 13/13 [00:00<00:00, 16334.92it/s]
Fetching 13 files: 100%|█████████████████████████████████████████████████████████████| 13/13 [00:00<00:00, 12451.69it/s]
Model error (ValueError): No chat template is set for this processor. Please either set the `chat_template` attribute, or provide a chat template as an argument. See https://huggingface.co/docs/transformers/main/en/chat_templating for more information.
Error processing model: mlx-community/gemma-3-12b-pt-8bit
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^


vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
Processing mlx-community/llava-v1.6-34b-8bit
Fetching 17 files: 100%|█████████████████████████████████████████████████████████████| 17/17 [00:00<00:00, 11939.58it/s]
Fetching 17 files: 100%|█████████████████████████████████████████████████████████████| 17/17 [00:00<00:00, 17710.67it/s]
==========
Files: ['/Users/jrp/Pictures/Processed/20250329-163408_DSC03146.jpg'] 

Prompt: <|im_start|>user
<image>
Provide a factual caption, description and comma-separated keywords or tags for this image so that it can be catalogued and searched for easily. The picture was taken in River Adur, Shoreham-by-Sea, England, United Kingdom, UK on 2025-03-29 16:34:08 from the GPS location +50.829800°, -0.272917°. Do not include this GPS location or the date in your response.<|im_end|>
<|im_start|>assistant

Warning: Failed to process inputs with error: unsupported operand type(s) for //: 'int' and 'NoneType' Trying to process inputs with return_tensors='pt'
Model error (ValueError): Failed to process inputs with error: unsupported operand type(s) for //: 'int' and 'NoneType'. Please install PyTorch and try again.
Error processing model: mlx-community/llava-v1.6-34b-8bit
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^


vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
Processing mlx-community/llava-v1.6-mistral-7b-8bit
Fetching 12 files: 100%|█████████████████████████████████████████████████████████████| 12/12 [00:00<00:00, 38333.32it/s]
Fetching 12 files: 100%|█████████████████████████████████████████████████████████████| 12/12 [00:00<00:00, 17307.99it/s]
==========
Files: ['/Users/jrp/Pictures/Processed/20250329-163408_DSC03146.jpg'] 

Prompt: [INST] <image>
Provide a factual caption, description and comma-separated keywords or tags for this image so that it can be catalogued and searched for easily. The picture was taken in River Adur, Shoreham-by-Sea, England, United Kingdom, UK on 2025-03-29 16:34:08 from the GPS location +50.829800°, -0.272917°. Do not include this GPS location or the date in your response. [/INST]
The sky is cloudy. 
==========
Prompt: 2289 tokens, 224.435 tokens-per-sec
Generation: 8 tokens, 44.030 tokens-per-sec
Peak memory: 12.019 GB


Model output:

The sky is cloudy. 

Generation time: 12.7s

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^


vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
Processing mlx-community/paligemma2-10b-ft-docci-448-6bit
Fetching 8 files: 100%|████████████████████████████████████████████████████████████████| 8/8 [00:00<00:00, 22857.24it/s]
Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.52, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`.
Fetching 8 files: 100%|███████████████████████████████████████████████████████████████| 8/8 [00:00<00:00, 103883.69it/s]
==========
Files: ['/Users/jrp/Pictures/Processed/20250329-163408_DSC03146.jpg'] 

Prompt: <image>Provide a factual caption, description and comma-separated keywords or tags for this image so that it can be catalogued and searched for easily. The picture was taken in River Adur, Shoreham-by-Sea, England, United Kingdom, UK on 2025-03-29 16:34:08 from the GPS location +50.829800°, -0.272917°. Do not include this GPS location or the date in your response.
A medium shot of two boats on the shore of a river. The boat on the left is white, and the boat on the right is blue. The boats are side by side, with the blue boat on the right. The boats are on a grassy area with puddles of water. The boats have white masts and are tied to the shore with a rope. There are more boats in the background on the shore. A building is in the background, and there are houses on the left and right sides of the building. The sky is gray and cloudy.
==========
Prompt: 1135 tokens, 478.054 tokens-per-sec
Generation: 112 tokens, 39.745 tokens-per-sec
Peak memory: 11.216 GB


Model output:

A medium shot of two boats on the shore of a river. The boat on the left is white, and the boat on the right is blue. The boats are side by side, with the blue boat on the right. The boats are on a grassy area with puddles of water. The boats have white masts and are tied to the shore with a rope. There are more boats in the background on the shore. A building is in the background, and there are houses on the left and right sides of the building. The sky is gray and cloudy.

Generation time: 8.4s

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^


vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
Processing mlx-community/paligemma2-10b-ft-docci-448-bf16
Fetching 10 files: 100%|██████████████████████████████████████████████████████████████| 10/10 [00:00<00:00, 8111.20it/s]
Fetching 10 files: 100%|█████████████████████████████████████████████████████████████| 10/10 [00:00<00:00, 19645.45it/s]
==========
Files: ['/Users/jrp/Pictures/Processed/20250329-163408_DSC03146.jpg'] 

Prompt: <image>Provide a factual caption, description and comma-separated keywords or tags for this image so that it can be catalogued and searched for easily. The picture was taken in River Adur, Shoreham-by-Sea, England, United Kingdom, UK on 2025-03-29 16:34:08 from the GPS location +50.829800°, -0.272917°. Do not include this GPS location or the date in your response.
A medium shot of two boats on the shore of a river. The boat on the right is blue with white detailing, and the boat on the left is white with blue detailing. The boats are side by side, with the blue boat on the right. The boats are on a grassy area with puddles of water. There are more boats in the background on the shore. There is a building in the background with a tower on the left side of it. There are houses in the background as well. The sky is gray and cloudy.
==========
Prompt: 1135 tokens, 424.574 tokens-per-sec
Generation: 108 tokens, 4.637 tokens-per-sec
Peak memory: 26.860 GB


Model output:

A medium shot of two boats on the shore of a river. The boat on the right is blue with white detailing, and the boat on the left is white with blue detailing. The boats are side by side, with the blue boat on the right. The boats are on a grassy area with puddles of water. There are more boats in the background on the shore. There is a building in the background with a tower on the left side of it. There are houses in the background as well. The sky is gray and cloudy.

Generation time: 31.0s

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^


vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
Processing mlx-community/paligemma2-3b-ft-docci-448-bf16
Fetching 8 files: 100%|████████████████████████████████████████████████████████████████| 8/8 [00:00<00:00, 23140.99it/s]
Fetching 8 files: 100%|████████████████████████████████████████████████████████████████| 8/8 [00:00<00:00, 27481.11it/s]
==========
Files: ['/Users/jrp/Pictures/Processed/20250329-163408_DSC03146.jpg'] 

Prompt: <image>Provide a factual caption, description and comma-separated keywords or tags for this image so that it can be catalogued and searched for easily. The picture was taken in River Adur, Shoreham-by-Sea, England, United Kingdom, UK on 2025-03-29 16:34:08 from the GPS location +50.829800°, -0.272917°. Do not include this GPS location or the date in your response.
A medium-close-up view of two boats that are facing away from each other. The boat on the left is white, and along the bottom portion of it there are yellow markings. The boat on the right is blue, and along the bottom portion of it there are black markings. The boat on the left is being held up by a metal pole, while the one on the right is being held up by a wooden pole. The boats are sitting on a land that is made up of grass, and along the grass there are small puddles of water. Behind the boats, there is a building that is made up of glass and is rectangular. The top portion of the building is black, while the bottom portion is white. Behind the building, there is a tree line that is made up of trees that are green. The sky is light blue and is full of white clouds.
==========
Prompt: 1135 tokens, 1323.506 tokens-per-sec
Generation: 178 tokens, 15.972 tokens-per-sec
Peak memory: 11.080 GB


Model output:

A medium-close-up view of two boats that are facing away from each other. The boat on the left is white, and along the bottom portion of it there are yellow markings. The boat on the right is blue, and along the bottom portion of it there are black markings. The boat on the left is being held up by a metal pole, while the one on the right is being held up by a wooden pole. The boats are sitting on a land that is made up of grass, and along the grass there are small puddles of water. Behind the boats, there is a building that is made up of glass and is rectangular. The top portion of the building is black, while the bottom portion is white. Behind the building, there is a tree line that is made up of trees that are green. The sky is light blue and is full of white clouds.

Generation time: 14.9s

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^


vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
Processing mlx-community/paligemma2-3b-pt-896-4bit
Fetching 7 files: 100%|████████████████████████████████████████████████████████████████| 7/7 [00:00<00:00, 15220.39it/s]
Fetching 7 files: 100%|████████████████████████████████████████████████████████████████| 7/7 [00:00<00:00, 14034.48it/s]
==========
Files: ['/Users/jrp/Pictures/Processed/20250329-163408_DSC03146.jpg'] 

Prompt: <image>Provide a factual caption, description and comma-separated keywords or tags for this image so that it can be catalogued and searched for easily. The picture was taken in River Adur, Shoreham-by-Sea, England, United Kingdom, UK on 2025-03-29 16:34:08 from the GPS location +50.829800°, -0.272917°. Do not include this GPS location or the date in your response.
boats on the shore
==========
Prompt: 4207 tokens, 1205.674 tokens-per-sec
Generation: 5 tokens, 123.535 tokens-per-sec
Peak memory: 8.392 GB


Model output:

boats on the shore

Generation time: 5.8s

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^


vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
Processing mlx-community/pixtral-12b-8bit
Fetching 11 files: 100%|█████████████████████████████████████████████████████████████| 11/11 [00:00<00:00, 60787.01it/s]
Fetching 11 files: 100%|██████████████████████████████████████████████████████████████| 11/11 [00:00<00:00, 5729.21it/s]
==========
Files: ['/Users/jrp/Pictures/Processed/20250329-163408_DSC03146.jpg'] 

Prompt: <s>[INST][IMG]Provide a factual caption, description and comma-separated keywords or tags for this image so that it can be catalogued and searched for easily. The picture was taken in River Adur, Shoreham-by-Sea, England, United Kingdom, UK on 2025-03-29 16:34:08 from the GPS location +50.829800°, -0.272917°. Do not include this GPS location or the date in your response.[/INST]
**Caption:**
A serene scene of sailboats resting on the muddy banks of the River Adur in Shoreham-by-Sea, with the town's colorful buildings and a prominent clock tower in the background.

**Description:**
The image captures a tranquil riverside setting in Shoreham-by-Sea. Two sailboats, one white and the other blue, are beached on the muddy riverbank. The boats are equipped with tall masts and ropes, suggesting they are prepared for sailing. In the background, a variety of colorful buildings line the river, including a notable clock tower that stands out against the cloudy sky. The overall atmosphere is calm and picturesque, reflecting the quiet charm of this coastal town.

**Keywords:**
Sailboats, River Adur, Shoreham-by-Sea, Clock tower, Muddy riverbank, Coastal town, Boats, Sailing, Ropes, Masts, Buildings, Cloudy sky, Tranquil, Picturesque, England, United Kingdom, UK
==========
Prompt: 2928 tokens, 370.174 tokens-per-sec
Generation: 207 tokens, 27.309 tokens-per-sec
Peak memory: 17.233 GB


Model output:

**Caption:**
A serene scene of sailboats resting on the muddy banks of the River Adur in Shoreham-by-Sea, with the town's colorful buildings and a prominent clock tower in the background.

**Description:**
The image captures a tranquil riverside setting in Shoreham-by-Sea. Two sailboats, one white and the other blue, are beached on the muddy riverbank. The boats are equipped with tall masts and ropes, suggesting they are prepared for sailing. In the background, a variety of colorful buildings line the river, including a notable clock tower that stands out against the cloudy sky. The overall atmosphere is calm and picturesque, reflecting the quiet charm of this coastal town.

**Keywords:**
Sailboats, River Adur, Shoreham-by-Sea, Clock tower, Muddy riverbank, Coastal town, Boats, Sailing, Ropes, Masts, Buildings, Cloudy sky, Tranquil, Picturesque, England, United Kingdom, UK

Generation time: 18.9s

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^


vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
Processing mlx-community/pixtral-12b-bf16
Fetching 13 files: 100%|█████████████████████████████████████████████████████████████| 13/13 [00:00<00:00, 27847.78it/s]
Fetching 13 files: 100%|█████████████████████████████████████████████████████████████| 13/13 [00:00<00:00, 16518.01it/s]
==========
Files: ['/Users/jrp/Pictures/Processed/20250329-163408_DSC03146.jpg'] 

Prompt: <s>[INST][IMG]Provide a factual caption, description and comma-separated keywords or tags for this image so that it can be catalogued and searched for easily. The picture was taken in River Adur, Shoreham-by-Sea, England, United Kingdom, UK on 2025-03-29 16:34:08 from the GPS location +50.829800°, -0.272917°. Do not include this GPS location or the date in your response.[/INST]
**Caption:** Two sailboats resting on the muddy shore of River Adur in Shoreham-by-Sea, with a backdrop of colorful buildings and a cloudy sky.

**Description:** The image captures a serene scene at the River Adur in Shoreham-by-Sea. Two sailboats, one white and the other blue, are beached on the muddy shore. In the background, a row of colorful buildings lines the riverbank, including a prominent modern building with a distinctive design. The sky is overcast, adding a calm and tranquil atmosphere to the scene.

**Keywords:** Sailboats, River Adur, Shoreham-by-Sea, Muddy shore, Colorful buildings, Overcast sky, Maritime scene, England, United Kingdom, UK, Boats, Waterfront, Cloudy sky, Maritime, Nautical, River, Mudflat, Seaside, Coastal, Sailing, Vessels, Boating, Marina, Dock, Harbor, Moored, Anchored, Still water, Tranquil, Peaceful, Calm, Serene, Picturesque, Scenic, Landscape, Architecture, Urban, Rural, Residential, Buildings, Houses, Homes, Dwellings, Structures, Water, Sky, Clouds, Weather, Atmosphere, Environment, Nature, Outdoors, Leisure, Recreation, Pastime, Hobbies, Sailing boats, Yachts, Vessels, Maritime activities, Coastal living, Seaside town, Fishing village, Harbor town, Waterfront town, River town, Coastal town, Seaside community, Boating community, Sailing community, Nautical community, Maritime community, Waterfront community, River community, Coastal community, Seaside life, Boating life, Sailing life, Nautical life, Maritime life, Waterfront life, River life, Coastal life, Seaside lifestyle, Boating lifestyle, Sailing lifestyle, Nautical lifestyle, Maritime lifestyle, Waterfront lifestyle, River lifestyle, Coastal lifestyle, Seaside culture, Boating culture, Sailing culture, Nautical culture, Maritime culture, Waterfront culture, River culture, Coastal culture, Seaside tradition, Boating tradition, Sailing tradition, Nautical tradition, Maritime tradition, Waterfront tradition, River tradition, Coastal tradition, Seaside heritage, Boating heritage, Sailing heritage, Nautical heritage, Maritime heritage, Waterfront heritage, River heritage, Coastal
==========
Prompt: 2928 tokens, 363.985 tokens-per-sec
Generation: 500 tokens, 3.809 tokens-per-sec
Peak memory: 32.669 GB


Model output:

**Caption:** Two sailboats resting on the muddy shore of River Adur in Shoreham-by-Sea, with a backdrop of colorful buildings and a cloudy sky.

**Description:** The image captures a serene scene at the River Adur in Shoreham-by-Sea. Two sailboats, one white and the other blue, are beached on the muddy shore. In the background, a row of colorful buildings lines the riverbank, including a prominent modern building with a distinctive design. The sky is overcast, adding a calm and tranquil atmosphere to the scene.

**Keywords:** Sailboats, River Adur, Shoreham-by-Sea, Muddy shore, Colorful buildings, Overcast sky, Maritime scene, England, United Kingdom, UK, Boats, Waterfront, Cloudy sky, Maritime, Nautical, River, Mudflat, Seaside, Coastal, Sailing, Vessels, Boating, Marina, Dock, Harbor, Moored, Anchored, Still water, Tranquil, Peaceful, Calm, Serene, Picturesque, Scenic, Landscape, Architecture, Urban, Rural, Residential, Buildings, Houses, Homes, Dwellings, Structures, Water, Sky, Clouds, Weather, Atmosphere, Environment, Nature, Outdoors, Leisure, Recreation, Pastime, Hobbies, Sailing boats, Yachts, Vessels, Maritime activities, Coastal living, Seaside town, Fishing village, Harbor town, Waterfront town, River town, Coastal town, Seaside community, Boating community, Sailing community, Nautical community, Maritime community, Waterfront community, River community, Coastal community, Seaside life, Boating life, Sailing life, Nautical life, Maritime life, Waterfront life, River life, Coastal life, Seaside lifestyle, Boating lifestyle, Sailing lifestyle, Nautical lifestyle, Maritime lifestyle, Waterfront lifestyle, River lifestyle, Coastal lifestyle, Seaside culture, Boating culture, Sailing culture, Nautical culture, Maritime culture, Waterfront culture, River culture, Coastal culture, Seaside tradition, Boating tradition, Sailing tradition, Nautical tradition, Maritime tradition, Waterfront tradition, River tradition, Coastal tradition, Seaside heritage, Boating heritage, Sailing heritage, Nautical heritage, Maritime heritage, Waterfront heritage, River heritage, Coastal

Generation time: 144.6s

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^


╔══════════════════════════════════╦═══════════╦═══════════╦═══════════╦═══════════╗
║ Model                            ║  Active   ║   Cache   ║   Peak    ║   Time    ║
╠══════════════════════════════════╬═══════════╬═══════════╬═══════════╬═══════════╣
║ SmolVLM2-2.2B-Instruct-mlx       ║   4,336MB ║   2,171MB ║   5,192MB ║     3.0s  ║
║ SmolVLM-Instruct                 ║   4,346MB ║   2,159MB ║   5,191MB ║     3.2s  ║
║ SmolVLM-Instruct-bf16            ║   4,346MB ║   2,156MB ║   5,191MB ║     3.5s  ║
║ Florence-2-large-ft-bf16         ║   2,048MB ║  10,760MB ║   3,297MB ║     4.8s  ║
║ paligemma2-3b-pt-896-4bit        ║   1,709MB ║   9,310MB ║   8,003MB ║     5.8s  ║
║ paligemma2-10b-ft-docci-448-6b   ║   7,606MB ║   4,140MB ║  10,696MB ║     8.4s  ║
║ Idefics3-8B-Llama3-bf16          ║  15,961MB ║   2,669MB ║  17,582MB ║     8.8s  ║
║ llava-v1.6-mistral-7b-8bit       ║   8,065MB ║   6,389MB ║  11,463MB ║    12.7s  ║
║ Phi-3.5-vision-instruct-bf16     ║   9,198MB ║   3,734MB ║  10,434MB ║    13.1s  ║
║ Phi-3.5-vision-instruct          ║   9,198MB ║   3,736MB ║  10,434MB ║    13.2s  ║
║ deepseek-vl2-8bit                ║  28,014MB ║   5,131MB ║  29,788MB ║    13.4s  ║
║ paligemma2-3b-ft-docci-448-bf1   ║   8,129MB ║     701MB ║  10,567MB ║    14.9s  ║
║ pixtral-12b-8bit                 ║  13,237MB ║   7,029MB ║  16,434MB ║    18.9s  ║
║ paligemma2-10b-ft-docci-448-bf   ║  22,642MB ║   4,986MB ║  25,616MB ║    31.0s  ║
║ Molmo-7B-D-0924-bf16             ║  15,312MB ║  33,104MB ║  46,325MB ║    43.1s  ║
║ Llama-3.2-11B-Vision-Instruct    ║  23,191MB ║   5,532MB ║  23,864MB ║    46.4s  ║
║ Llama-3.2-11B-Vision-Instruct-   ║  11,214MB ║   3,999MB ║  14,272MB ║    46.4s  ║
║ Molmo-7B-D-0924-8bit             ║   7,247MB ║  34,080MB ║  39,723MB ║    50.4s  ║
║ pixtral-12b-bf16                 ║  27,592MB ║   8,441MB ║  31,156MB ║   144.6s  ║
║ Llama-3.2-90B-Vision-Instruct-   ║  47,132MB ║   5,762MB ║  51,030MB ║   319.0s  ║
╠══════════════════════════════════╬═══════════╬═══════════╬═══════════╬═══════════╣
║ AVERAGE                          ║  13,526MB ║   7,799MB ║  51,030MB ║    40.2s  ║
╚══════════════════════════════════╩═══════════╩═══════════╩═══════════╩═══════════╝

Total execution time: 822.1s

[jrp2014]

This is the test harness I used:

#!/usr/bin/env python3
"""Image analysis and caption generation using MLX Vision Language Models."""

# Standard library imports
from argparse import ArgumentParser, Namespace
import subprocess
import time
from contextlib import contextmanager
from dataclasses import dataclass
from datetime import datetime
from functools import lru_cache
from pathlib import Path
from typing import (
    Any, Dict, Final, Generator, List, Optional, NamedTuple, TypedDict,
    Union
)

# Third-party imports
import mlx.core as mx
from PIL import Image, UnidentifiedImageError
from PIL.ExifTags import GPSTAGS, TAGS

# Local imports
from mlx_vlm import (
    load, generate, __version__ as vlm_version
)
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

# Type aliases and definitions
ExifDict = Dict[str, Any]  # EXIF data dictionary
MetadataDict = Dict[str, str]  # Metadata results dictionary
PathLike = Union[str, Path]

# Constants - Defaults
DEFAULT_MAX_TOKENS: Final[int] = 500
DEFAULT_FOLDER: Final[Path] = Path("/Users/jrp/Pictures/Processed")

# Constants - EXIF and IPTC
IMPORTANT_EXIF_TAGS: Final[frozenset[str]] = frozenset({
    "DateTimeOriginal", "ImageDescription", "CreateDate",
    "Make", "Model", "LensModel", "ExposureTime",
    "FNumber", "ISOSpeedRatings", "FocalLength", "ExposureProgram",
})

DATE_FORMATS: Final[tuple[str, ...]] = (
    "%Y:%m:%d %H:%M:%S",
    "%Y-%m-%d %H:%M:%S",
    "%Y%m%d",
)

EXIF_DATE_TAGS: Final[tuple[str, ...]] = (
    "DateTimeOriginal",
    "CreateDate",
    "DateTime"
)

IPTC_TAGS: Final[dict[str, int]] = {
    "LOCATION": 0x5A,
    "DATECREATED": 0x37,
    "CITY": 0x5A,
    "COUNTRY": 0x65,
}

# Type definitions

class MemoryStats(NamedTuple):
    """Memory statistics container."""
    active: float    # Active memory in MB
    cached: float    # Cache memory in MB
    peak: float      # Peak memory in MB
    time: float      # Processing time in seconds

    @staticmethod
    def zero() -> 'MemoryStats':
        """Create a zero-initialized MemoryStats object."""
        return MemoryStats(active=0.0, cached=0.0, peak=0.0, time=0.0)

@dataclass(frozen=True)
class ModelResult:
    """Container for model processing results."""
    model_name: str
    output: str
    stats: MemoryStats

class ModelConfig(TypedDict):
    """Type hints for model configuration."""
    chat_template: str
    image_token: str
    eos_token: str

# Memory management
@contextmanager
def memory_tracker(verbose: bool = False) -> Generator[None, None, None]:
    """Context manager to track MLX memory usage and timing."""
    mx.clear_cache()
    mx.reset_peak_memory()
    yield

# File handling functions
def get_most_recently_modified_file(folder: PathLike) -> Optional[PathLike]:
    """Return the name of the most recently modified file in the given folder, excluding hidden files."""
    try:
        folder_path: Path = Path(folder)
        if not folder_path.is_dir():
            return None
        files: List[str] = sorted(
            (f for f in folder_path.iterdir() if not f.name.startswith(".")),
            key=lambda f: f.stat().st_mtime,
            reverse=True,
        )
        return files[0] if files else None
    except Exception as e:
        print(f"Error while getting the most recently modified file: {e}")
        return None


def print_image_dimensions(image_path: PathLike, debug: bool = False) -> None:
    """Print the dimensions of the image given as a path string."""
    try:
        with Image.open(image_path) as img:
            width, height = img.size
            mpx: float = width * height / 1_000_000
            if debug:
                print(f"Image dimensions: {width}x{height} ({mpx:.1f} MPixels)")
    except Exception as e:
        print(f"Error while getting image dimensions: {e}")


def _map_key(k: int) -> str:
    try:
        return TAGS[k]
    except KeyError:
        return GPSTAGS.get(k, k)


@lru_cache(maxsize=128)
def get_exif(image_path: PathLike) -> Optional[ExifDict]:
    """Get EXIF data from image with caching."""
    try:
        with Image.open(image_path) as img:
            if not hasattr(img, '_getexif') or not (exif := img._getexif()):
                return None
            return {
                TAGS.get(tag_id, tag_id): value
                for tag_id, value in exif.items()
            }
    except (UnidentifiedImageError, OSError) as e:
        print(f"Error reading image file: {e}")
        return None
    except Exception as e:
        print(f"Unexpected error reading EXIF data: {e}")
        return None


def format_date(date_str: str) -> str:
    """Format date string in a consistent way."""
    try:
        for fmt in DATE_FORMATS:
            try:
                return datetime.strptime(date_str, fmt).strftime("%Y-%m-%d %H:%M:%S")
            except ValueError:
                continue
        return date_str
    except Exception:
        return "Unknown date"


def extract_image_metadata(
    image_path: PathLike, 
    verbose: bool = False,
    debug: bool = False,
) -> MetadataDict:
    """Extract and format image metadata with improved error handling."""
    metadata: MetadataDict = {
        "GPS": "Unknown location",
        "date": "Unknown date",
        "description": "No description",
    }

    try:
        exif = get_exif(image_path)
        if not exif:
            return metadata

        # Extract description
        if desc := exif.get("ImageDescription"):
            metadata["description"] = str(desc).strip(", ")

        # Handle GPS data
        if gps_info := exif.get("GPSInfo"):
            try:
                if debug:
                    print("\nRaw GPS Info:")
                    for k, v in gps_info.items():
                        tag = GPSTAGS.get(k, k)
                        print(f"  {tag}: {v}")

                # Helper function for GPS coordinate conversion
                def get_coordinate(ref_tag: int, coord_tag: int) -> Optional[float]:
                    try:
                        ref = gps_info.get(ref_tag, 'N' if ref_tag == 1 else 'E')
                        if isinstance(ref, bytes):
                            ref = ref.decode()
                        
                        coords = gps_info.get(coord_tag)
                        if not coords or len(coords) != 3:
                            return None

                        degrees = coords[0][0] / coords[0][1] if isinstance(coords[0], tuple) else float(coords[0])
                        minutes = coords[1][0] / coords[1][1] if isinstance(coords[1], tuple) else float(coords[1])
                        seconds = coords[2][0] / coords[2][1] if isinstance(coords[2], tuple) else float(coords[2])

                        decimal = degrees + (minutes / 60.0) + (seconds / 3600.0)
                        return -decimal if ref in ['S', 'W'] else decimal
                    except (IndexError, TypeError, ZeroDivisionError) as e:
                        if debug:
                            print(f"Error converting coordinate: {e}")
                        return None

                # Extract coordinates
                lat = get_coordinate(1, 2)  # 1=GPSLatitudeRef, 2=GPSLatitude
                lon = get_coordinate(3, 4)  # 3=GPSLongitudeRef, 4=GPSLongitude

                if debug:
                    print(f"Extracted coordinates - Latitude: {lat}, Longitude: {lon}")

                if lat is not None and lon is not None:
                    metadata["GPS"] = f"{lat:+.6f}°, {lon:+.6f}°"
                    if debug:
                        print(f"Final GPS string: {metadata['GPS']}")

            except Exception as exc:
                print(f"GPS parsing error: {exc}")
                if debug:
                    import traceback
                    traceback.print_exc()

        # Extract and format date
        for tag in EXIF_DATE_TAGS:
            if date_str := exif.get(tag):
                metadata["date"] = format_date(str(date_str))
                break

        return metadata

    except Exception as exc:
        print(f"Error extracting metadata: {exc}")
        if debug:
            import traceback
            traceback.print_exc()
        return metadata


def pretty_print_exif(exif: ExifDict, verbose: bool = False) -> None:
    """Pretty print EXIF data in a formatted table."""
    if not exif:
        print("No EXIF data found")
        return

    print("\n╔════════════════════════════════╦═══════════════════════════════════════════════╗")
    print("║ EXIF Tag                       ║ Value                                         ║")
    print("╠════════════════════════════════╬═══════════════════════════════════════════════╣")

    for tag, value in sorted(exif.items()):
        # Skip binary data and GPS info (handled separately)
        if isinstance(value, bytes) or tag == "GPSInfo":
            continue

        # Skip if not important and not verbose
        if not verbose and tag not in IMPORTANT_EXIF_TAGS:
            continue

        # Format the tag and value strings
        tag_str = str(tag)[:30]
        value_str = str(value)[:45]

        # Highlight important tags
        if tag in IMPORTANT_EXIF_TAGS:
            print(f"║ \033[1m{tag_str:<30}\033[0m ║ {value_str:<45} ║")
        else:
            print(f"║ {tag_str:<30} ║ {value_str:<45} ║")

    print("╚════════════════════════════════╩═══════════════════════════════════════════════╝")


# Model handling functions
def get_model_names() -> List[str]:
    """Get list of available models from huggingface cache."""
    try:
        output: bytes = subprocess.check_output(
            [
                "/opt/homebrew/Caskroom/miniconda/base/envs/mlx/bin/huggingface-cli",
                "scan-cache",
            ],
            stderr=subprocess.PIPE,
        )
        return sorted(
            line.split()[0] 
            for line in output.decode("utf-8").split("\n")[2:-4]
            if line.strip()
        )
    except subprocess.CalledProcessError as e:
        print(f"Error accessing huggingface cache: {e}")
        return []


def print_model_stats(results: List[ModelResult]) -> None:
    """Print a table of model statistics."""
    if not results:
        return

    print("\n╔══════════════════════════════════╦═══════════╦═══════════╦═══════════╦═══════════╗")
    print("║ Model                            ║  Active   ║   Cache   ║   Peak    ║   Time    ║")
    print("╠══════════════════════════════════╬═══════════╬═══════════╬═══════════╬═══════════╣")
    
    for result in sorted(results, key=lambda x: x.stats.time):
        model_name = result.model_name.split('/')[-1][:30]
        print(
            f"║ {model_name:<32} ║"
            f"{result.stats.active:>8,.0f}MB ║"
            f"{result.stats.cached:>8,.0f}MB ║"
            f"{result.stats.peak:>8,.0f}MB ║"
            f"{result.stats.time:>8.1f}s  ║"
        )
    
    if len(results) > 0:
        avg_stats = MemoryStats(
            active=sum(r.stats.active for r in results) / len(results),
            cached=sum(r.stats.cached for r in results) / len(results),
            peak=max(r.stats.peak for r in results),
            time=sum(r.stats.time for r in results) / len(results)
        )
        print("╠══════════════════════════════════╬═══════════╬═══════════╬═══════════╬═══════════╣")
        print(
             f"║ AVERAGE                          ║"
            f"{avg_stats.active:>8,.0f}MB ║"
            f"{avg_stats.cached:>8,.0f}MB ║"
            f"{avg_stats.peak:>8,.0f}MB ║"
            f"{avg_stats.time:>8.1f}s  ║"
        )
    
    print("╚══════════════════════════════════╩═══════════╩═══════════╩═══════════╩═══════════╝")

def process_model(
    model_path: PathLike, 
    image_path: PathLike, 
    prompt: str, 
    verbose: bool, 
    debug: bool = False,
    max_tokens: int = DEFAULT_MAX_TOKENS
) -> Optional[ModelResult]:
    """Process model with MLX memory tracking and error handling."""
    model: mx.Model = None
    tokenizer: mx.Tokenizer = None
    
    try:
        initial_mem = mx.get_active_memory() / 1024 / 1024
        initial_cache = mx.get_cache_memory() / 1024 / 1024
        start_time = time.perf_counter()

        with memory_tracker():
            if not Path(image_path).exists():
                raise FileNotFoundError(f"Image file not found: {image_path}")

            trust_remote_code: bool = True
            model, tokenizer = load(model_path, trust_remote_code=trust_remote_code)
            config: ModelConfig = load_config(model_path, trust_remote_code=trust_remote_code)

            formatted_prompt: str = apply_chat_template(
                tokenizer, config, prompt, num_images=1
            )

            output: str = generate(
                model, tokenizer, formatted_prompt, [image_path], 
                verbose=verbose, max_tokens=max_tokens
            )

            # Force sync and measure final memory state
            mx.eval(model.parameters())
            memory_stats = MemoryStats(
                active=mx.get_active_memory() / 1024 / 1024 - initial_mem,
                cached=mx.get_cache_memory() / 1024 / 1024 - initial_cache,
                peak=mx.get_peak_memory() / 1024 / 1024,
                time=time.perf_counter() - start_time
            )

            if debug:
                print(f"\nModel memory usage:")
                print(f"  Active:  {memory_stats.active:>8,.0f}MB")
                print(f"  Cached:  {memory_stats.cached:>8,.0f}MB")
                print(f"  Peak:    {memory_stats.peak:>8,.0f}MB")
                print(f"  Runtime: {memory_stats.time:>8.1f}s")

            return ModelResult(
                model_name=model_path,
                output=output,
                stats=memory_stats
            )
    except FileNotFoundError as e:
        print(f"File not found: {e}")
    except Exception as exc:
        print(f"Model error ({type(exc).__name__}): {exc}")
        if debug:
            import traceback
            traceback.print_exc()
    finally:
        if model is not None:
            del model
        if tokenizer is not None:
            del tokenizer
        mx.clear_cache()
        mx.reset_peak_memory()
    
    return None

def main(
    folder: PathLike,
    prompt: Optional[str] = None,
    verbose: bool = False,
    max_tokens: int = DEFAULT_MAX_TOKENS,
    debug: bool = False,
) -> None:
    """Process images with MLX vision language models."""
    start_time: float = time.perf_counter()
    results: List[ModelResult] = []

    if not Path(folder).is_dir():
        print(f"Error: {folder} is not a valid directory")
        return

    if not (most_recent_file := get_most_recently_modified_file(folder)):
        print("No files found in the folder.")
        return

    print(f"\nProcessing file: {most_recent_file}")
    print_image_dimensions(most_recent_file, debug=debug)

    metadata = extract_image_metadata(most_recent_file, verbose, debug)
    if not metadata:
        print("Error: Could not extract metadata from image")
        return

    if debug:
        print(
            f"Lat & Longitude: {metadata['GPS']}\n"
            f"Date:            {metadata['date']}\n"
            f"Description:     {metadata['description']}"
        )

    results : list[ModelResult] = []

    default_prompt: str = (
        "Provide a factual caption, description and comma-separated "
        "keywords or tags for this image so that it can be catalogued "
        "and searched for easily. The picture was taken in "
        f"{metadata['description']} on {metadata['date']}"
    )
    if metadata['GPS'] != "Unknown location":
        default_prompt += (
            f" from the GPS location {metadata['GPS']}. "
            "Do not include this GPS location or the date in your response."
        )

    actual_prompt: str = default_prompt if prompt is None else prompt

    models: List[str] = get_model_names()
    if not models:
        print("No models found in huggingface cache")
        return

    for model_path in models:
        print(f"\n{80 * 'v'}")
        print(f"\033[1mProcessing {model_path}\033[0m")

        if debug:
            print(f"Model: {model_path}")
            print(f"Image: {most_recent_file}")
            print(f"Prompt: {actual_prompt}")
            print("╔══════════════════════════════════════════════════════════════════════╗")

        if result := process_model(
            model_path, most_recent_file.as_posix(), actual_prompt, verbose, debug, max_tokens
        ):
            if debug:
                print(f"Result:\n{result}\n")
            print("\n\nModel output:\n")
            print(result.output)
            print(f"\nGeneration time: {result.stats.time:.1f}s\n")
            results.append(result)
            if debug:
                print_model_stats(results)
        else: 
            print(f"Error processing model: {model_path}")
                
        print(f"{80 * '^'}\n")

 
    if results:
        print_model_stats(results)

    elapsed: float = time.perf_counter() - start_time
    print(f"\nTotal execution time: {elapsed:.1f}s")


if __name__ == "__main__":
    print(f"\nMLX version: {mx.__version__}")
    print(f"MLX-VLM version: {vlm_version}\n")

    parser: ArgumentParser = ArgumentParser(
        description="Describe, caption and keyword the most recently modified image in a folder"
    )
    parser.add_argument(
        "-f",
        "--folder",
        type=PathLike,
        default=DEFAULT_FOLDER,
        help="The folder to scan for the most recently modified file",
    )
    parser.add_argument(
        "-v",
        "--verbose",
        action="store_true",
        help="Enable verbose output",
    )
    parser.add_argument(
        "-p",
        "--prompt",
        type=str,
        help="Custom prompt to use for image analysis",
    )
    parser.add_argument(
        "-m",
        "--max-tokens",
        type=int,
        default=DEFAULT_MAX_TOKENS,
        help=f"Maximum number of tokens to generate (default: {DEFAULT_MAX_TOKENS})",
    )
    parser.add_argument(
        "-d",
        "--debug",
        action="store_true",
        help="Enable debug output",
    )

    args: Namespace = parser.parse_args()
    main(args.folder, args.prompt, args.verbose, args.max_tokens, args.debug)

[jrp2014]

Seems to work OK now. This script https://gist.github.com/jrp2014/e57b8132fe319e3c264c20a9b8a9fe5b produces a coloured version of this:

--- Model Performance Summary (Console) ---
╔══════════════════════════════════════╤══════════════╤══════════════╤══════════════╤══════════════╗
║ Model                                │     Active Δ │      Cache Δ │     Peak Mem │         Time ║
╠══════════════════════════════════════╪══════════════╪══════════════╪══════════════╪══════════════╣
║ SmolVLM-Instruct-bf16                │     4,508 MB │       166 MB │     4,656 MB │       1.79 s ║
║ SmolVLM-Instruct                     │     4,508 MB │       166 MB │     4,656 MB │       1.85 s ║
║ Florence-2-large-ft-bf16             │     1,646 MB │     1,561 MB │     1,707 MB │       2.53 s ║
║ llava-v1.6-mistral-7b-8bit           │     8,056 MB │       308 MB │     8,264 MB │       3.91 s ║
║ paligemma2-10b-ft-docci-448-6bit     │     7,522 MB │       792 MB │     8,286 MB │       4.25 s ║
║ paligemma2-3b-pt-896-4bit            │     1,717 MB │       549 MB │     2,040 MB │       4.80 s ║
║ paligemma2-10b-ft-docci-448-bf16     │    19,328 MB │       277 MB │    19,588 MB │       5.11 s ║
║ Idefics3-8B-Llama3-bf16              │    16,928 MB │       154 MB │    17,067 MB │       5.13 s ║
║ SmolVLM2-2.2B-Instruct-mlx           │     4,501 MB │       445 MB │     4,719 MB │       5.21 s ║
║ Llama-3.2-11B-Vision-Instruct-8bit   │    11,358 MB │       357 MB │    11,606 MB │       7.87 s ║
║ paligemma2-3b-ft-docci-448-bf16      │     6,068 MB │       403 MB │     6,292 MB │       9.22 s ║
║ pixtral-12b-8bit                     │    13,488 MB │       339 MB │    13,723 MB │       9.34 s ║
║ Molmo-7B-D-0924-8bit                 │     9,047 MB │     1,479 MB │    10,443 MB │       9.37 s ║
║ deepseek-vl2-8bit                    │    29,337 MB │       967 MB │    29,898 MB │      13.36 s ║
║ Molmo-7B-D-0924-bf16                 │    16,044 MB │     1,342 MB │    17,318 MB │      14.55 s ║
║ pixtral-12b-bf16                     │    25,368 MB │       241 MB │    25,507 MB │      14.71 s ║
║ Llama-3.2-11B-Vision-Instruct        │    21,343 MB │       245 MB │    21,503 MB │      14.80 s ║
║ Phi-3.5-vision-instruct              │     9,153 MB │     2,646 MB │    10,799 MB │      20.38 s ║
║ Phi-3.5-vision-instruct-bf16         │     8,788 MB │     3,013 MB │    10,799 MB │      20.97 s ║
║ llava-v1.6-34b-8bit                  │    36,933 MB │       443 MB │    37,207 MB │      23.03 s ║
║ Llama-3.2-90B-Vision-Instruct-4bit   │    49,850 MB │       582 MB │    50,210 MB │      35.41 s ║
║ Florence-2-large-ft (Failed: load)   │            - │            - │            - │            - ║
║ QwQ-32B-4bit (Failed: load)          │            - │            - │            - │            - ║
║ QwQ-32B-8bit (Failed: load)          │            - │            - │            - │            - ║
║ gemma-3-12b-pt-8bit (Failed: prompt) │            - │            - │            - │            - ║
╠══════════════════════════════════════╪══════════════╪══════════════╪══════════════╪══════════════╣
║ AVG/PEAK (21 Success)                │    14,547 MB │       785 MB │    50,210 MB │      10.84 s ║
╚══════════════════════════════════════╧══════════════╧══════════════╧══════════════╧══════════════╝

It also outputs the results to an HTML table, with the actual results, which makes it easier to judge the quality of the output.

[jrp2014]

My script has started to produce hallucinations.

The results that I get seem similar to when I run the mlx-vlm test_smoke script with language only:

python  test_smoke.py --models-file models.txt --image ~/Pictures/Processed/20250412-160700_DSC03314.jpg --language-only-prompt "Provide factual caption, description, keywords/tags for cataloguing/searching. taken around 2025-04-13 22:05:21 Focus on visual content. Avoid repeating context unless visible." 

  0%|                                                                                            | 0/27 [00:00<?, ?it/s]╭──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ Testing HuggingFaceTB/SmolVLM-Instruct                                                                               │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
Loading model...
Fetching 12 files: 100%|██████████████████████████████████████████████████████████████| 12/12 [00:00<00:00, 8842.52it/s]
Fetching 12 files: 100%|█████████████████████████████████████████████████████████████| 12/12 [00:00<00:00, 33893.37it/s]
✓ Model loaded successfully in 0.71 seconds                                                      | 0/12 [00:00<?, ?it/s]


Testing vision-language generation...
==========
Files: ['/Users/jrp/Pictures/Processed/20250412-160700_DSC03314.jpg'] 

Prompt: <|im_start|>User:<image>Describe this image.<end_of_utterance>
Assistant:
 A narrow street with a brick building on the left and a white building on the right. The street is made of asphalt and has yellow lines marking the center. The buildings are old and have different colors and styles. The brick building on the left has a large window and a door. The white building on the right has a large window and a door.
==========
Prompt: 1560 tokens, 900.275 tokens-per-sec
Generation: 72 tokens, 115.807 tokens-per-sec
Peak memory: 5.735 GB
✓ vision-language generation successful


Testing language-only generation...
==========
Files: None 

Prompt: <|im_start|>User: Provide factual caption, description, keywords/tags for cataloguing/searching. taken around 2025-04-13 22:05:21 Focus on visual content. Avoid repeating context unless visible.<end_of_utterance>
Assistant:
 A man and woman are standing in front of a large, white, rectangular structure. The man is wearing a white shirt and black pants, while the woman is wearing a white dress. The structure is made of white panels and has a large, circular opening in the front. The man and woman are standing in front of the structure, and the woman is looking at the man. The man is looking at the woman. The structure is in a field, and there are trees and grass in the background
==========
Prompt: 57 tokens, 1480.329 tokens-per-sec
Generation: 100 tokens, 124.964 tokens-per-sec
Peak memory: 4.584 GB
✓ language-only generation successful


Cleaning up...
✓ Cleanup complete

The vision language version is fine, the language only, is completely made up.

My script, which used to produce good enough responses, is

#!/usr/bin/env python3
"""Image analysis and caption generation using MLX Vision Language Models."""

# Standard library imports
import argparse
import contextlib
import html
import logging
import platform
import re  # For ANSI code stripping
import signal
import subprocess
import sys
import time
import traceback
import types # Added for TracebackType
from dataclasses import dataclass, field
from datetime import datetime
from functools import lru_cache
from pathlib import Path
from typing import (
    Any, Dict, Final, List, NamedTuple, NoReturn,
    Optional, TextIO, Tuple, Union, Callable,
    TypeVar, cast
)
import os # Needed for os.path.getmtime

from huggingface_hub import HFCacheInfo, scan_cache_dir
from huggingface_hub import __version__ as hf_version
from huggingface_hub.errors import HFValidationError

# Third-party imports
try:
    import mlx.core as mx
except ImportError:
    print("Core dependency missing: mlx. Please install it.", file=sys.stderr)
    sys.exit(1)

try:
    from PIL import Image, UnidentifiedImageError
    from PIL.ExifTags import GPSTAGS, TAGS
    pillow_version = Image.__version__ if hasattr(Image, '__version__') else 'N/A'
except ImportError:
    print("Error: Pillow not found. Please install it (`pip install Pillow`).", file=sys.stderr)
    pillow_version = "N/A"
    sys.exit(1)

# Local application/library specific imports
try:
    from mlx_vlm import (__version__ as vlm_version, generate, load)
    from mlx_vlm.prompt_utils import apply_chat_template
    from mlx_vlm.utils import load_config
except ImportError:
    print("Error: mlx-vlm not found. Please install it (`pip install mlx-vlm`).", file=sys.stderr)
    sys.exit(1)

# Optional imports for version reporting
try:
    from mlx_lm import __version__ as mlx_lm_version
except ImportError:
    mlx_lm_version = "N/A"
try:
    import transformers
    transformers_version = transformers.__version__
except ImportError:
    transformers_version = "N/A"

# Custom timeout context manager (for Python < 3.11)
# Note: This implementation relies on signal.SIGALRM and will not work on Windows.
class timeout(contextlib.ContextDecorator):
    def __init__(self, seconds: float) -> None:
        self.seconds: float = seconds
        self.timer: Optional[Callable[[int, Optional[types.FrameType]], Any]] = None # Use FrameType

    def _timeout_handler(self, signum: int, frame: Optional[types.FrameType]) -> NoReturn: # Use FrameType
        raise TimeoutError(f"Operation timed out after {self.seconds} seconds")

    def __enter__(self) -> 'timeout':
        # Check if SIGALRM is available (won't be on Windows)
        if hasattr(signal, 'SIGALRM'):
            if self.seconds > 0:
                try:
                    self.timer = signal.signal(signal.SIGALRM, self._timeout_handler)
                    signal.alarm(int(self.seconds))
                except ValueError as e:
                    # Running in a thread or environment where signals are restricted
                    logger.warning(f"Could not set SIGALRM for timeout: {e}. Timeout disabled.")
                    self.seconds = 0 # Disable timeout functionality
        else:
            if self.seconds > 0:
                logger.warning("Timeout functionality requires signal.SIGALRM, not available on this platform (e.g., Windows). Timeout disabled.")
                self.seconds = 0 # Disable timeout functionality
        return self

    def __exit__(self, exc_type: Optional[type[BaseException]],
                 exc_val: Optional[BaseException],
                 exc_tb: Optional[types.TracebackType]) -> None: # Use TracebackType
        # Only try to reset the alarm if it was successfully set
        if hasattr(signal, 'SIGALRM') and self.seconds > 0 and self.timer is not None:
            signal.alarm(0)
            signal.signal(signal.SIGALRM, self.timer)


# Configure logging
logger = logging.getLogger(__name__)
# BasicConfig called in main()


# --- Version Info ---
def get_library_versions() -> Dict[str, str]:
    """Collect versions of key libraries."""
    versions = {
        'mlx': getattr(mx, '__version__', 'N/A'),
        'mlx-vlm': vlm_version if 'vlm_version' in globals() else 'N/A',
        'mlx-lm': mlx_lm_version,
        'huggingface-hub': hf_version,
        'transformers': transformers_version,
        'Pillow': pillow_version
    }
    return versions

def print_version_info(versions: Dict[str, str]) -> None:
    """Print collected library versions to the console with date."""
    print("\n--- Library Versions ---")
    max_len = max(len(k) for k in versions) + 1 if versions else 10
    for name, ver in sorted(versions.items()):
        status_color = Colors.GREEN if ver != "N/A" else Colors.YELLOW
        name_padded = name.ljust(max_len)
        print(f"{name_padded}: {Colors.colored(ver, status_color)}")
    print(f"\nGenerated: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}")


# --- ANSI Color Codes for Console Output ---
class Colors:
    """ANSI color codes for terminal output"""
    RESET: Final[str] = "\033[0m"
    BOLD: Final[str] = "\033[1m"
    RED: Final[str] = "\033[91m"
    GREEN: Final[str] = "\033[92m"
    YELLOW: Final[str] = "\033[93m"
    BLUE: Final[str] = "\033[94m"
    MAGENTA: Final[str] = "\033[95m"
    CYAN: Final[str] = "\033[96m"
    WHITE: Final[str] = "\033[97m"
    GRAY: Final[str] = "\033[90m"
    _enabled: bool = sys.stderr.isatty()
    _ansi_escape_re: re.Pattern[str] = re.compile(r'\x1B(?:[@-Z\\-_]|\[[0-?]*[ -/]*[@-~])')

    @staticmethod
    def colored(text: str, color: str) -> str:
        if not Colors._enabled:
            return text
        return f"{color}{text}{Colors.RESET}"

    @staticmethod
    def visual_len(text: Union[str, Any]) -> int:
        if not isinstance(text, str):
             text = str(text)
        return len(Colors._ansi_escape_re.sub('', text))


# Type aliases and definitions
T = TypeVar('T')
ExifValue = Any
ExifDict = Dict[Union[str, int], ExifValue]
MetadataDict = Dict[str, str]
PathLike = Union[str, Path]
GPSTupleElement = Union[int, float]
GPSTuple = Tuple[GPSTupleElement, GPSTupleElement, GPSTupleElement]

# Constants - Defaults
DEFAULT_MAX_TOKENS: Final[int] = 500
DEFAULT_FOLDER: Final[Path] = Path.home() / "Pictures" / "Processed"
DEFAULT_HTML_OUTPUT: Final[Path] = Path("results.html")
DEFAULT_TEMPERATURE: Final[float] = 0.5

# Constants - EXIF
IMPORTANT_EXIF_TAGS: Final[frozenset[str]] = frozenset({
    "DateTimeOriginal", "ImageDescription", "CreateDate", "Make", "Model",
    "LensModel", "ExposureTime", "FNumber", "ISOSpeedRatings",
    "FocalLength", "ExposureProgram",
})
DATE_FORMATS: Final[Tuple[str, ...]] = ("%Y:%m:%d %H:%M:%S", "%Y-%m-%d %H:%M:%S", "%Y%m%d")
EXIF_DATE_TAGS: Final[Tuple[str, ...]] = ("DateTimeOriginal", "CreateDate",  "DateTime")
GPS_LAT_REF_TAG: Final[int] = 1
GPS_LAT_TAG: Final[int] = 2
GPS_LON_REF_TAG: Final[int] = 3
GPS_LON_TAG: Final[int] = 4
GPS_INFO_TAG_ID: Final[int] = 34853  # Standard EXIF tag ID for GPS IFD

# Type definitions
class MemoryStats(NamedTuple):
    """Memory statistics container (values represent deltas or peak)."""
    active: float
    cached: float
    peak: float
    time: float
    @staticmethod
    def zero() -> 'MemoryStats': return MemoryStats(0.0, 0.0, 0.0, 0.0)

@dataclass(frozen=True)
class ModelResult:
    """Container for model processing results, including failures."""
    model_name: str
    success: bool
    output: Optional[str] = None
    stats: MemoryStats = field(default_factory=MemoryStats.zero)
    error_stage: Optional[str] = None
    error_message: Optional[str] = None
    captured_output_on_fail: Optional[str] = None


# --- File Handling ---
def find_most_recent_file(folder: Path) -> Optional[Path]:
    """Return the Path of the most recently modified file in the folder."""
    if not folder.is_dir():
        logger.error(Colors.colored(f"Provided path is not a directory: {folder}", Colors.RED))
        return None
    try:
        files: List[Path] = [
            f for f in folder.iterdir() if f.is_file() and not f.name.startswith(".")
        ]
        if not files:
            logger.warning(Colors.colored(f"No non-hidden files found in: {folder}", Colors.YELLOW))
            return None
        files.sort(key=lambda f: f.stat().st_mtime, reverse=True)
        most_recent: Path = files[0]
        logger.debug(f"Most recent file found: {most_recent}")
        return most_recent
    except PermissionError:
        logger.error(Colors.colored(f"Permission denied accessing folder: {folder}", Colors.RED))
        return None
    except OSError as e:
        logger.error(Colors.colored(f"OS error scanning folder {folder}: {e}", Colors.RED))
        return None

def print_image_dimensions(image_path: Path) -> None:
    """Print the dimensions and megapixel count of the image."""
    try:
        with Image.open(image_path) as img:
            width, height = img.size
            mpx: float = (width * height) / 1_000_000
            print(f"Image dimensions: {Colors.colored(f'{width}x{height}', Colors.CYAN)} ({Colors.colored(f'{mpx:.1f}', Colors.CYAN)} MPixels)")
    except FileNotFoundError:
        logger.error(Colors.colored(f"Image file not found: {image_path}", Colors.RED))
    except UnidentifiedImageError:
        logger.error(Colors.colored(f"Cannot identify image file (may be corrupt or wrong format): {image_path}", Colors.RED))
    except Exception as e:
        logger.error(Colors.colored(f"Error reading image dimensions for {image_path}: {e}", Colors.RED))


# --- EXIF & Metadata Handling ---
@lru_cache(maxsize=128)
def get_exif_data(image_path: Path) -> Optional[ExifDict]:
    """Extract EXIF data from an image file, including decoding GPS IFD."""
    # GPS_INFO_TAG_ID moved to global constants

    try:
        with Image.open(image_path) as img:
            exif_raw: Any = img.getexif()
            if not exif_raw:
                logger.debug(f"No EXIF data found in {image_path}")
                return None

            # First pass: decode main EXIF tags
            exif_decoded: ExifDict = {}
            
            # Try to get DateTimeOriginal directly from IFD using tag ID
            dt_original_tag_id = 36867  # Standard EXIF tag ID for DateTimeOriginal
            if dt_original_tag_id in exif_raw:
                exif_decoded["DateTimeOriginal"] = exif_raw[dt_original_tag_id]

            # Decode remaining EXIF tags
            for tag_id, value in exif_raw.items():
                if tag_id == GPS_INFO_TAG_ID:
                    continue
                if tag_id == dt_original_tag_id:
                    continue
                tag_name = TAGS.get(tag_id, str(tag_id))
                exif_decoded[tag_name] = value

            logger.debug(f"EXIF data decoded for {image_path}: {exif_decoded}")

            # Second pass: handle GPS IFD specifically
            if GPS_INFO_TAG_ID in exif_raw:
                try:
                    gps_ifd = exif_raw.get_ifd(GPS_INFO_TAG_ID)
                    if isinstance(gps_ifd, dict) and gps_ifd:
                        gps_decoded = {}
                        for gps_tag_id, gps_value in gps_ifd.items():
                            gps_tag_name = GPSTAGS.get(gps_tag_id, str(gps_tag_id))
                            gps_decoded[gps_tag_name] = gps_value
                        exif_decoded["GPSInfo"] = gps_decoded
                except Exception as e:
                    # Add image path to the warning log
                    logger.warning(f"Failed to decode GPS data for {image_path}: {e}")

            return exif_decoded

    except (FileNotFoundError, UnidentifiedImageError) as e:
        logger.error(f"Error reading image file: {e}")
    except Exception as e:
        logger.error(f"Unexpected error reading EXIF: {e}")
    return None

def _format_exif_date(date_str_input: Any) -> Optional[str]:
    """Attempt to parse and format a date string from EXIF."""
    date_str: str
    if not isinstance(date_str_input, str):
        try:
            date_str = str(date_str_input)
        except Exception:
            logger.debug(f"Could not convert potential date value '{date_str_input}' to string.")
            return None
    else:
        date_str = date_str_input

    for fmt in DATE_FORMATS:
        try:
            dt_obj: datetime = datetime.strptime(date_str, fmt)
            return dt_obj.strftime("%Y-%m-%d %H:%M:%S")
        except (ValueError, TypeError):
            continue  # Try next format

    logger.debug(f"Could not parse date string '{date_str}' with known formats.")
    return None


def _convert_gps_coordinate(ref: Optional[Union[str, bytes]], coord: Any) -> Optional[float]:
    """Convert various GPS coordinate formats to decimal degrees."""
    if not ref or not coord:
        return None

    try:
        # Handle reference direction
        ref_str = ref.decode('ascii') if isinstance(ref, bytes) else str(ref)
        ref_upper = ref_str.upper()
        if ref_upper not in ['N', 'S', 'E', 'W']:
            return None

        # Handle different coordinate formats
        if isinstance(coord, tuple) and len(coord) == 3:
            # Standard DMS format
            degrees, minutes, seconds = [
                float(v.numerator)/float(v.denominator) 
                if hasattr(v, 'numerator') else float(v) 
                for v in coord
            ]
        elif isinstance(coord, (list, tuple)) and len(coord) == 2:
            # Decimal degrees and minutes format
            degrees, minutes = coord
            degrees = float(degrees.numerator)/float(degrees.denominator) if hasattr(degrees, 'numerator') else float(degrees)
            minutes = float(minutes.numerator)/float(minutes.denominator) if hasattr(minutes, 'numerator') else float(minutes)
            seconds = 0.0
        else:
            # Try direct decimal degrees
            try:
                return -float(coord) if ref_upper in ['S', 'W'] else float(coord)
            except (TypeError, ValueError):
                return None

        # Validate ranges
        if not (0 <= degrees <= 180 and 0 <= minutes < 60 and 0 <= seconds < 60):
            return None

        decimal = degrees + (minutes / 60.0) + (seconds / 3600.0)
        return -decimal if ref_upper in ['S', 'W'] else decimal

    except (ValueError, TypeError, ZeroDivisionError) as e:
        logger.debug(f"GPS conversion error: {type(e).__name__}: {e}")
        return None


def extract_image_metadata(image_path: Path) -> MetadataDict:
    """Extract key metadata: date, GPS, and selected EXIF tags.

    Prioritizes DateTimeOriginal from raw IFD, then falls back to other date sources.
    """
    metadata: MetadataDict = {}
    exif_data: Optional[ExifDict] = get_exif_data(image_path)

    # 1. Try to get date from EXIF, prioritizing DateTimeOriginal from raw IFD
    if exif_data:
        dt_original = exif_data.get("DateTimeOriginal")
        if dt_original:
            formatted_date = _format_exif_date(dt_original)
            if formatted_date:
                metadata['date'] = formatted_date
                metadata['date_source'] = "DateTimeOriginal (IFD)"
                metadata['date_tag'] = "DateTimeOriginal"
                logger.debug(f"Using DateTimeOriginal from IFD: {formatted_date}")
                
        # Only try other date tags if DateTimeOriginal wasn't found
        if 'date' not in metadata:
            for tag in EXIF_DATE_TAGS:
                if tag in exif_data:
                    date_str = str(exif_data[tag])
                    formatted_date = _format_exif_date(date_str)
                    if formatted_date:
                        metadata['date'] = formatted_date
                        metadata['date_source'] = f"EXIF ({tag})"
                        metadata['date_tag'] = tag
                        logger.debug(f"Using date from {tag}: {formatted_date}")
                        break

    # 2. Fallback to file modification time if no EXIF date found
    if 'date' not in metadata:
        try:
            mtime = os.path.getmtime(image_path)
            metadata['date'] = datetime.fromtimestamp(mtime).strftime("%Y-%m-%d %H:%M:%S")
            metadata['date_source'] = "File Modification Time"
            metadata['date_tag'] = "mtime"
            logger.debug(f"Using file modification time: {metadata['date']}")
        except OSError as e:
            logger.warning(f"Could not get modification time: {e}")
            metadata['date'] = "Unknown"
            metadata['date_source'] = "Unavailable"
            metadata['date_tag'] = None
            
    # 3. Extract GPS coordinates if available
    if exif_data and "GPSInfo" in exif_data and isinstance(exif_data["GPSInfo"], dict):
        gps_info = exif_data["GPSInfo"]
        lat = gps_info.get("GPSLatitude")
        lat_ref = gps_info.get("GPSLatitudeRef")
        lon = gps_info.get("GPSLongitude")
        lon_ref = gps_info.get("GPSLongitudeRef")

        if lat and lat_ref and lon and lon_ref:
            try:
                # Ensure values are tuples of numbers before conversion
                if (
                    isinstance(lat, tuple) and len(lat) == 3 and all(isinstance(x, (int, float)) for x in lat) and
                    isinstance(lon, tuple) and len(lon) == 3 and all(isinstance(x, (int, float)) for x in lon) and
                    isinstance(lat_ref, str) and isinstance(lon_ref, str)
                ):
                    # Cast to expected tuple types for _convert_gps
                    lat_tuple = cast(GPSTuple, lat)
                    lon_tuple = cast(GPSTuple, lon)

                    latitude = _convert_gps(lat_tuple, lat_ref)
                    longitude = _convert_gps(lon_tuple, lon_ref)
                    metadata['gps'] = f"{latitude:.6f}, {longitude:.6f}"
                    logger.debug(f"Extracted GPS {metadata['gps']} for {image_path.name}")
                else:
                     logger.warning(f"Unexpected GPS data format in {image_path.name}: lat={lat}, lon={lon}")
            except (ValueError, TypeError, ZeroDivisionError) as e:
                logger.warning(f"Could not decode GPS coordinates for {image_path.name}: {e}")
        else:
            logger.debug(f"Incomplete GPS tags found for {image_path.name}")
    else:
        logger.debug(f"No GPSInfo found in EXIF for {image_path.name}")

    # 4. Add other important EXIF tags
    if exif_data:
        for tag in IMPORTANT_EXIF_TAGS:
            # Avoid overwriting the date we just determined
            if tag not in EXIF_DATE_TAGS and tag in exif_data:
                value = exif_data[tag]
                # Simple conversion for common types, avoid complex objects
                if isinstance(value, (str, int, float)):
                     metadata[tag] = str(value).strip()
                elif isinstance(value, bytes):
                     # Attempt to decode bytes, fallback to repr
                     try:
                         metadata[tag] = value.decode('utf-8', errors='replace').strip()
                     except UnicodeDecodeError:
                         metadata[tag] = repr(value)
                elif isinstance(value, tuple):
                     metadata[tag] = ", ".join(map(str, value))
                # Add other simple types if needed, but avoid deep structures

    return metadata

def _convert_gps(coord: GPSTuple, ref: str) -> float:
    """Convert EXIF GPS coordinate (degrees, minutes, seconds) to decimal degrees."""
    degrees, minutes, seconds = coord
    decimal_degrees = degrees + (minutes / 60.0) + (seconds / 3600.0)
    if ref in ['S', 'W']:
        decimal_degrees = -decimal_degrees
    return decimal_degrees

def pretty_print_exif(exif: ExifDict, verbose: bool = False) -> None:
    """Pretty print key EXIF data in a formatted table, using colors."""
    # (Implementation remains the same as previous correct version)
    if not exif:
        print("No EXIF data available.")
        return

    print(f"\n--- {Colors.colored('Key EXIF Data', Colors.CYAN)} ---")
    tags_to_print: List[Tuple[str, str, bool]] = []
    for tag, value in exif.items():
        tag_str = str(tag)
        if tag_str == "GPSInfo" and isinstance(value, dict):
            continue
        if isinstance(value, dict):
            logger.debug(f"Skipping dictionary value for EXIF tag '{tag_str}' in pretty print.")
            continue
        value_str: str
        if isinstance(value, bytes):
             try:
                 decoded_str = value.decode('utf-8', errors='replace')
                 value_str = decoded_str[:57] + "..." if len(decoded_str) > 60 else decoded_str
             except Exception:
                 value_str = f"<bytes len={len(value)}>"
        elif isinstance(value, tuple) and len(value) > 10:
            value_str = f"<tuple len={len(value)}>"
        else:
             try:
                  value_str = str(value)
                  if len(value_str) > 60:
                      value_str = value_str[:57] + "..."
             except Exception as str_err:
                 # Log the specific error during string conversion
                 logger.debug(f"Could not convert EXIF value for tag '{tag_str}' to string: {str_err}")
                 value_str = f"<unrepresentable type: {type(value).__name__}>"

        is_important = tag_str in IMPORTANT_EXIF_TAGS
        if verbose or is_important:
            tags_to_print.append((tag_str, value_str, is_important))

    if not tags_to_print:
        print("No relevant EXIF tags found to display.")
        return
    tags_to_print.sort(key=lambda x: x[0])
    max_tag_len = max(Colors.visual_len(t[0]) for t in tags_to_print) if tags_to_print else 20
    max_val_len = max(Colors.visual_len(t[1]) for t in tags_to_print) if tags_to_print else 40
    min_width = 10
    max_tag_len = max(max_tag_len, min_width)
    max_val_len = max(max_val_len, min_width + 5)
    header_color = Colors.BLUE
    border_color = Colors.BLUE
    important_color = Colors.YELLOW
    def pad(text: str, width: int, left: bool = True) -> str:
        pad_len = max(0, width - Colors.visual_len(text))
        return f"{text}{' '*pad_len}" if left else f"{' '*pad_len}{text}"
    print(Colors.colored(f"╔{'═' * (max_tag_len + 2)}╤{'═' * (max_val_len + 2)}╗", border_color))
    print(f"{Colors.colored('║', border_color)} {pad(Colors.colored('Tag', header_color), max_tag_len)} {Colors.colored('│', border_color)} {pad(Colors.colored('Value', header_color), max_val_len)} {Colors.colored('║', border_color)}")
    print(Colors.colored(f"╠{'═' * (max_tag_len + 2)}╪{'═' * (max_val_len + 2)}╣", border_color))
    for tag_name, value_display, is_important_tag in tags_to_print:
        tag_display = Colors.colored(tag_name, Colors.BOLD + important_color) if is_important_tag else tag_name
        print(f"{Colors.colored('║', border_color)} {pad(tag_display, max_tag_len)} {Colors.colored('│', border_color)} {pad(value_display, max_val_len)} {Colors.colored('║', border_color)}")
    print(Colors.colored(f"╚{'═' * (max_tag_len + 2)}╧{'═' * (max_val_len + 2)}╝", border_color))


# --- Model Handling ---
def get_cached_model_ids() -> List[str]:
    """Get list of model repo IDs from the huggingface cache."""
    if scan_cache_dir is None:
        logger.error(Colors.colored("huggingface_hub library not found. Cannot scan Hugging Face cache.", Colors.RED))
        return []
    try:
        # Use CacheInfo (public class)
        logger.debug("Scanning Hugging Face cache directory...")
        cache_info: HFCacheInfo = scan_cache_dir()
        model_ids = sorted([repo.repo_id for repo in cache_info.repos])
        logger.debug(f"Found {len(model_ids)} potential models in Hugging Face cache: {model_ids}")
        return model_ids
    except HFValidationError:
        logger.error(Colors.colored("Hugging Face cache directory invalid.", Colors.RED))
        return []
    except FileNotFoundError:
        logger.error(Colors.colored("Hugging Face cache directory not found.", Colors.RED))
        return []
    except Exception as e:
        logger.error(Colors.colored(f"Unexpected error scanning Hugging Face cache: {type(e).__name__}: {e}", Colors.RED), exc_info=logger.level <= logging.DEBUG)
        return []


def print_model_stats(results: List[ModelResult]) -> None:
    """Print a table summarizing model performance statistics to the console, including failures."""
    # (Implementation remains the same as previous correct version)
    if not results:
         print("No model results to display.")
         return
    results.sort(key=lambda x: (not x.success, x.stats.time if x.success else 0))
    display_names = [(r.model_name.split('/')[-1]) for r in results]
    max_name_len_base = max(len(name) for name in display_names) if display_names else 20
    max_name_len_cap = 44
    max_name_len_base = min(max_name_len_base, max_name_len_cap)
    col_width = 12
    header_color = Colors.BLUE
    border_color = Colors.BLUE
    summary_color = Colors.YELLOW
    failure_color = Colors.RED
    failure_text_color = Colors.GRAY
    captured_marker_plain = "(+cap)"
    captured_marker_colored = Colors.colored(captured_marker_plain, Colors.GRAY)
    max_len_needed = 0
    temp_display_names = []
    for i, r in enumerate(results):
        name = display_names[i]
        current_max_name = max_name_len_base
        if len(name) > current_max_name:
            name = name[:current_max_name - 3] + "..."
        display_str = name
        if not r.success:
            fail_info_text = f" (Failed: {r.error_stage or '?'})"
            if r.captured_output_on_fail: 
                fail_info_text += f" {captured_marker_plain}"
            display_str += fail_info_text
        temp_display_names.append(display_str)
        max_len_needed = max(max_len_needed, len(display_str))
    max_name_len = min(max(max_name_len_base, max_len_needed), 55)
    def pad(text: str, width: int, left: bool = True) -> str:
        vlen = Colors.visual_len(text)
        padding = ' ' * max(0, width - vlen)
        return f"{text}{padding}" if left else f"{padding}{text}"
    print(f"\n--- {Colors.colored('Model Performance Summary (Console)', Colors.CYAN)} ---")
    print(Colors.colored(f"╔{'═' * (max_name_len + 2)}╤{'═' * (col_width + 2)}╤{'═' * (col_width + 2)}╤{'═' * (col_width + 2)}╤{'═' * (col_width + 2)}╗", border_color))
    header_model = pad(Colors.colored('Model', header_color), max_name_len)
    header_active = pad(Colors.colored('Active Δ', header_color), col_width, False)
    header_cache = pad(Colors.colored('Cache Δ', header_color), col_width, False)
    header_peak = pad(Colors.colored('Peak Mem', header_color), col_width, False)
    header_time = pad(Colors.colored('Time', header_color), col_width, False)
    print(f"{Colors.colored('║', border_color)} {header_model} {Colors.colored('│', border_color)} {header_active} {Colors.colored('│', border_color)} {header_cache} {Colors.colored('│', border_color)} {header_peak} {Colors.colored('│', border_color)} {header_time} {Colors.colored('║', border_color)}")
    print(Colors.colored(f"╠{'═' * (max_name_len + 2)}╪{'═' * (col_width + 2)}╪{'═' * (col_width + 2)}╪{'═' * (col_width + 2)}╪{'═' * (col_width + 2)}╣", border_color))
    successful_results = [r for r in results if r.success]
    for i, result in enumerate(results):
        model_disp_name_raw = display_names[i]
        model_display_text: str
        active_str: str
        cache_str: str
        peak_str: str
        time_str: str
        allowance = 15 if not result.success else 0
        if len(model_disp_name_raw) > max_name_len - allowance:
             truncate_at = max(0, max_name_len - allowance - 3)
             model_disp_name_raw = model_disp_name_raw[:truncate_at] + "..."
        if result.success:
            model_display_text = model_disp_name_raw
            active_str = f"{result.stats.active:,.0f} MB"
            cache_str = f"{result.stats.cached:,.0f} MB" 
            peak_str = f"{result.stats.peak:,.0f} MB"
            time_str = f"{result.stats.time:.2f} s"
        else:
            fail_info = f" (Failed: {result.error_stage or '?'})"
            if result.captured_output_on_fail: 
                fail_info += f" {captured_marker_colored}"
            model_display_text_base = model_disp_name_raw + fail_info.replace(captured_marker_colored, "(+cap)")
            model_display_text = Colors.colored(model_display_text_base, failure_color)
            if result.captured_output_on_fail:
                model_display_text = model_display_text.replace("(+cap)", captured_marker_colored)
            active_str = Colors.colored("-".rjust(col_width - 4), failure_text_color)
            cache_str = Colors.colored("-".rjust(col_width - 4), failure_text_color) 
            peak_str = Colors.colored("-".rjust(col_width - 4), failure_text_color) 
            time_str = Colors.colored("-".rjust(col_width - 2), failure_text_color)
        print(f"{Colors.colored('║', border_color)} {pad(model_display_text, max_name_len)} {Colors.colored('│', border_color)} {pad(active_str, col_width, False)} {Colors.colored('│', border_color)} {pad(cache_str, col_width, False)} {Colors.colored('│', border_color)} {pad(peak_str, col_width, False)} {Colors.colored('│', border_color)} {pad(time_str, col_width, False)} {Colors.colored('║', border_color)}")
    if successful_results:
        avg_active = sum(r.stats.active for r in successful_results) / len(successful_results)
        avg_cache = sum(r.stats.cached for r in successful_results) / len(successful_results)
        max_peak = max(r.stats.peak for r in successful_results)
        avg_time = sum(r.stats.time for r in successful_results) / len(successful_results)
        avg_active_str = f"{avg_active:,.0f} MB"
        avg_cache_str = f"{avg_cache:,.0f} MB"
        max_peak_str = f"{max_peak:,.0f} MB"
        avg_time_str = f"{avg_time:.2f} s"
        print(Colors.colored(f"╠{'═' * (max_name_len + 2)}╪{'═' * (col_width + 2)}╪{'═' * (col_width + 2)}╪{'═' * (col_width + 2)}╪{'═' * (col_width + 2)}╣", border_color))
        summary_title = f"AVG/PEAK ({len(successful_results)} Success)"
        summary_model = pad(Colors.colored(summary_title, summary_color), max_name_len)
        summary_active = pad(Colors.colored(avg_active_str, summary_color), col_width, False)
        summary_cache = pad(Colors.colored(avg_cache_str, summary_color), col_width, False)
        summary_peak = pad(Colors.colored(max_peak_str, summary_color), col_width, False)
        summary_time = pad(Colors.colored(avg_time_str, summary_color), col_width, False)
        print(f"{Colors.colored('║', border_color)} {summary_model} {Colors.colored('│', border_color)} {summary_active} {Colors.colored('│', border_color)} {summary_cache} {Colors.colored('│', border_color)} {summary_peak} {Colors.colored('│', border_color)} {summary_time} {Colors.colored('║', border_color)}")
    print(Colors.colored(f"╚{'═' * (max_name_len + 2)}╧{'═' * (col_width + 2)}╧{'═' * (col_width + 2)}╧{'═' * (col_width + 2)}╧{'═' * (col_width + 2)}╝", border_color))


# --- HTML Report Generation ---
def generate_html_report(results: List[ModelResult], filename: Path, versions: Dict[str, str]) -> None:
    """Generates an HTML file with model stats, output/errors, failures, and versions."""
    if not results:
        logger.warning("No results to generate HTML report.")
        return

    results.sort(key=lambda x: (not x.success, x.stats.time if x.success else 0))

    html_start = """
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Model Performance Results</title>
    <style>
        body { font-family: 'Segoe UI', Tahoma, Geneva, Verdana, sans-serif; margin: 20px; background-color: #f8f9fa; color: #212529; line-height: 1.6; }
        h1 { text-align: center; color: #343a40; border-bottom: 2px solid #dee2e6; padding-bottom: 10px; margin-bottom: 30px; }
        table { border-collapse: collapse; width: 95%; margin: 30px auto; box-shadow: 0 4px 8px rgba(0,0,0,0.1); background-color: #ffffff; }
        th, td { border: 1px solid #dee2e6; padding: 12px 15px; text-align: left; vertical-align: top; }
        th { background-color: #e9ecef; font-weight: 600; color: #495057; position: sticky; top: 0; z-index: 1; }
        tr:nth-child(even):not(.failed-row) { background-color: #f8f9fa; }
        tr:not(.failed-row):hover { background-color: #e2e6ea; }
        td.numeric, th.numeric { text-align: right; font-variant-numeric: tabular-nums; }
        .summary td { font-weight: bold; background-color: #d1ecf1; color: #0c5460; border-color: #bee5eb; }
        caption { caption-side: bottom; padding: 15px; font-style: italic; color: #6c757d; text-align: center; }
        .model-name { font-family: 'Courier New', Courier, monospace; font-weight: 500; }
        .model-output, .captured-output pre { white-space: pre-wrap; word-wrap: break-word; max-width: 600px; font-size: 0.9em; }
        .captured-output { background-color: #fff3cd; border-left: 3px solid #ffeeba; padding: 8px; margin-top: 10px; color: #856404; }
        .captured-output strong { color: #856404; }
        .captured-output pre { margin: 0; padding: 5px; background-color: transparent; border: none; color: #856404; }
        tr.failed-row { background-color: #f8d7da !important; color: #721c24; }
        tr.failed-row:hover { background-color: #f5c6cb !important; }
        tr.failed-row .model-output { font-style: italic; color: #721c24; }
        tr.failed-row td.numeric { color: #721c24; font-style: italic; }
        .error-message { font-weight: bold; display: block; color: #721c24; }
        footer { margin-top: 30px; text-align: center; font-size: 0.85em; color: #6c757d; }
        footer h2 { font-size: 1.1em; color: #495057; margin-bottom: 10px;}
        footer ul { list-style: none; padding: 0; margin: 0 0 10px 0; }
        footer li { display: inline-block; margin: 0 10px; }
        footer code { background-color: #e9ecef; padding: 2px 4px; border-radius: 3px; }
        footer p { margin-top: 5px; }
    </style>
</head>
<body>
    <h1>Model Performance Summary</h1>
    <table>
        <caption>Performance metrics and output/errors for Vision Language Model processing. Generated on """ + datetime.now().strftime('%Y-%m-%d %H:%M:%S') + """. Failures shown but excluded from averages.</caption>
        <thead>
            <tr>
                <th>Model</th>
                <th class="numeric">Active Δ (MB)</th> <th class="numeric">Cache Δ (MB)</th>
                <th class="numeric">Peak Mem (MB)</th> <th class="numeric">Time (s)</th>
                <th>Result / Error / Captured Output</th>
            </tr>
        </thead>
        <tbody>
"""
    html_rows = ""
    successful_results = [r for r in results if r.success]

    for result in results:
        model_disp_name = html.escape(result.model_name)
        row_class = ""
        result_content = ""
        stats_cells = ""

        if result.success:
            escaped_output = html.escape(result.output or "")
            result_content = f'<div class="model-output">{escaped_output}</div>'
            # Format stats as comma-separated integers for HTML
            stats_cells = f"""
                <td class="numeric">{result.stats.active:,.0f}</td>
                <td class="numeric">{result.stats.cached:,.0f}</td>
                <td class="numeric">{result.stats.peak:,.0f}</td>
                <td class="numeric">{result.stats.time:.2f}</td>
            """
        else:
            row_class = ' class="failed-row"'
            error_info = f"<span class='error-message'>Failed during '{html.escape(result.error_stage or 'Unknown')}'"
            if result.error_message:
                error_info += f": {html.escape(result.error_message)}"
            error_info += "</span>"
            result_content = error_info

            if result.captured_output_on_fail:
                escaped_capture = html.escape(result.captured_output_on_fail)
                # Wrap captured output in a distinct div/pre block
                result_content += f'<div class="captured-output"><strong>Captured Output (during generate):</strong><pre>{escaped_capture}</pre></div>'

            # Use hyphen placeholder for failed stats in HTML
            stats_cells = """
                <td class="numeric">-</td> <td class="numeric">-</td>
                <td class="numeric">-</td> <td class="numeric">-</td>
            """

        html_rows += f"""
            <!-- Data row for model: {model_disp_name} -->
            <tr{row_class}>
                <td class="model-name">{model_disp_name}</td>
                {stats_cells}
                <td>{result_content}</td>
            </tr>
"""

    html_summary_row = ""
    # Only show summary if there was at least one success
    if successful_results:
        avg_active = sum(r.stats.active for r in successful_results) / len(successful_results)
        avg_cache = sum(r.stats.cached for r in successful_results) / len(successful_results)
        max_peak = max(r.stats.peak for r in successful_results)
        avg_time = sum(r.stats.time for r in successful_results) / len(successful_results)
        summary_title = f"AVG/PEAK ({len(successful_results)} Success)"
        # Format summary stats as comma-separated integers
        html_summary_row = f"""
            <!-- Summary Row (Based on Successful Runs) -->
            <tr class="summary">
                <td>{summary_title}</td>
                <td class="numeric">{avg_active:,.0f}</td>
                <td class="numeric">{avg_cache:,.0f}</td>
                <td class="numeric">{max_peak:,.0f}</td>
                <td class="numeric">{avg_time:.2f}</td>
                <td></td>
            </tr>
"""

    # --- Add Version Info Footer ---
    html_footer = "<footer>\n<h2>Library Versions</h2>\n<ul>\n"
    # Use sorted items for consistent order in HTML
    for name, ver in sorted(versions.items()):
        status_style = 'color: green;' if ver != "N/A" else 'color: orange;'
        html_footer += f'<li>{html.escape(name)}: <code style="{status_style}">{html.escape(ver)}</code></li>\n'
    html_footer += "</ul>\n"
    # Add date to footer
    html_footer += f"<p>Report generated on: {datetime.now().strftime('%Y-%m-%d')}</p>\n</footer>"
    # -----------------------------

    html_end = f"""
        </tbody>
    </table>
    <!-- End of Table -->
    {html_footer}
</body>
</html>
"""
    html_content = html_start + html_rows + html_summary_row + html_end

    try:
        # *** Restore TextIO type hint for file handle 'f' ***
        f: TextIO
        with open(filename, "w", encoding="utf-8") as f:
            f.write(html_content)
        logger.info(f"HTML report saved to: {Colors.colored(str(filename.resolve()), Colors.GREEN)}")
    except IOError as e:
        logger.error(Colors.colored(f"Failed to write HTML report to {filename}: {e}", Colors.RED))
    except Exception as e:
         logger.error(Colors.colored(f"An unexpected error occurred while writing HTML report: {type(e).__name__}: {e}", Colors.RED), exc_info=logger.level <= logging.DEBUG)


def get_system_info() -> Tuple[str, str]:
    """Get system architecture and GPU information."""
    arch = platform.machine()
    gpu_info = "Unknown"
    try:
        # Try to get GPU info on macOS
        if platform.system() == "Darwin":
            result = subprocess.run(['system_profiler', 'SPDisplaysDataType'],
                                 capture_output=True, text=True, timeout=2)
            if result.returncode == 0:
                # Extract GPU info from system_profiler output
                gpu_lines = [line for line in result.stdout.split('\n')
                           if "Chipset Model:" in line]
                if gpu_lines:
                    gpu_info = gpu_lines[0].split("Chipset Model:")[-1].strip()
    except (subprocess.SubprocessError, TimeoutError):
        pass
    return arch, gpu_info

# --- Model Processing Core ---
def validate_inputs(image_path: PathLike, model_path: str, temperature: float = 0.0) -> None:
    """Validate input paths and parameters."""
    img_path = Path(image_path)
    if not img_path.exists():
        raise FileNotFoundError(f"Image not found: {img_path}")
    if not img_path.is_file():
        raise ValueError(f"Not a file: {img_path}")
    if img_path.suffix.lower() not in {'.jpg', '.jpeg', '.png', '.webp'}:
        raise ValueError(f"Unsupported image format: {img_path.suffix}")

    validate_temperature(temperature)

def validate_temperature(temp: float) -> None:
    """Validate temperature parameter is within acceptable range."""
    if not isinstance(temp, (int, float)):
        raise ValueError(f"Temperature must be a number, got {type(temp)}")
    if not 0.0 <= temp <= 1.0:
        raise ValueError(f"Temperature must be between 0 and 1, got {temp}")

def validate_image_accessible(image_path: Path) -> None:
    """Validate image file is accessible and supported."""
    try:
        with timeout(5):  # 5 second timeout for read test
            with Image.open(image_path) as img:
                img.verify()
    except TimeoutError:
        raise IOError(f"Timeout while reading image: {image_path}")
    except UnidentifiedImageError:
        raise ValueError(f"File is not a recognized image format: {image_path}")
    except Exception as e:
        raise IOError(f"Error accessing image {image_path}: {e}")

def process_image_with_model(
    model_identifier: str,
    image_path: Path,
    prompt: str,
    max_tokens: int,
    verbose: bool,
    trust_remote_code: bool = True,
    temperature: float = DEFAULT_TEMPERATURE,
) -> ModelResult:
    """Process an image with a Vision Language Model."""
    logger.info(f"Processing '{image_path.name}' with model: {model_identifier}")

    model = tokenizer = None
    arch, gpu_info = get_system_info()

    try:
        validate_temperature(temperature)
        validate_image_accessible(image_path)

        # Log system info in debug mode
        if logger.isEnabledFor(logging.DEBUG):
            logger.debug(f"System: {arch}, GPU: {gpu_info}")

        with timeout(30):  # 30 second timeout
            mx.clear_cache()
            mx.reset_peak_memory()

            initial_mem = mx.get_active_memory() / 1024 / 1024
            initial_cache = mx.get_cache_memory() / 1024 / 1024
            start_time = time.perf_counter()

            model, tokenizer = load(model_identifier, trust_remote_code=trust_remote_code)
            config = load_config(model_identifier, trust_remote_code=trust_remote_code)

            formatted_prompt = apply_chat_template(tokenizer, config, prompt, num_images=1)
            output = generate(
                model=model,
                processor=tokenizer,
                prompt=formatted_prompt,
                image_paths=[str(image_path)],
                max_tokens=max_tokens,
                verbose=verbose,
                temp=temperature
            )

            mx.eval(model.parameters())

            final_stats = MemoryStats(
                active=mx.get_active_memory() / 1024 / 1024 - initial_mem,
                cached=mx.get_cache_memory() / 1024 / 1024 - initial_cache,
                peak=mx.get_peak_memory() / 1024 / 1024,
                time=time.perf_counter() - start_time
            )

            return ModelResult(
                model_name=model_identifier,
                success=True,
                output=str(output) if output is not None else "",
                stats=final_stats
            )

    except TimeoutError:
        logger.error(f"Timeout while processing model {model_identifier}")
        return ModelResult(
            model_name=model_identifier,
            success=False,
            error_stage="timeout",
            error_message="Operation timed out"
        )
    except Exception as e:
        error_stage = "model_load" if model is None else "generate"
        logger.error(f"Failed during {error_stage}: {type(e).__name__}: {e}")
        if logger.isEnabledFor(logging.DEBUG):
            traceback.print_exc()
        return ModelResult(
            model_name=model_identifier,
            success=False,
            error_stage=error_stage,
            error_message=str(e)
        )
    finally:
        del model
        del tokenizer
        mx.clear_cache()
        mx.reset_peak_memory()


# --- Main Execution ---
def main(args: argparse.Namespace) -> None:
    """Main function to orchestrate image analysis."""
    # Configure logging level based on args
    log_level = logging.DEBUG if args.debug else (logging.INFO if args.verbose else logging.INFO)
    logging.basicConfig(level=log_level,
                        format='%(asctime)s - %(levelname)s - %(message)s',
                        handlers=[logging.StreamHandler(sys.stderr)], force=True)
    if args.debug:
        logger.debug("Debug mode enabled.")
    elif args.verbose:
        logger.info("Verbose mode enabled.")

    # Collect library versions early
    library_versions = get_library_versions()

    # Print library versions initially (now using the collected dict)
    try:
        # Use .get with default 'N/A' for safety
        print(f"MLX version: {Colors.colored(library_versions.get('mlx', 'N/A'), Colors.GREEN)}")
        print(f"MLX-VLM version: {Colors.colored(library_versions.get('mlx-vlm', 'N/A'), Colors.GREEN)}\n")
        # logger.debug(f"Default MLX device: {mx.default_device()}") # Optional: Keep only in debug
    except Exception as version_err:
        logger.warning(Colors.colored(f"Could not retrieve some library version info: {version_err}", Colors.YELLOW))

    # Warn about trusting remote code
    if args.trust_remote_code:
        logger.warning(Colors.colored("--- SECURITY WARNING ---", Colors.YELLOW + Colors.BOLD))
        logger.warning(Colors.colored("`--trust-remote-code` is enabled.", Colors.YELLOW))
        logger.warning(Colors.colored("-----------------------", Colors.YELLOW + Colors.BOLD))

    overall_start_time = time.perf_counter()

    # --- 1. Find Image ---
    folder_path = args.folder.resolve()
    logger.info(f"Scanning folder: {Colors.colored(str(folder_path), Colors.BLUE)}")
    if args.folder == DEFAULT_FOLDER and not DEFAULT_FOLDER.is_dir():
        print(Colors.colored(f"Warning: Default folder '{DEFAULT_FOLDER}' does not exist.", Colors.YELLOW), file=sys.stderr)
    image_path = find_most_recent_file(folder_path)
    if not image_path:
        print(Colors.colored(f"\nError: Could not find a suitable image file in {folder_path}.", Colors.RED), file=sys.stderr)
        sys.exit(1)
    resolved_image_path = image_path.resolve()
    print(f"\nProcessing file: {Colors.colored(resolved_image_path.name, Colors.MAGENTA)} (located at {resolved_image_path})")
    # Validate image readability early
    try:
        with Image.open(resolved_image_path) as img:
            img.verify() # Verify basic structure without loading full data
        print_image_dimensions(resolved_image_path)
    except (FileNotFoundError, UnidentifiedImageError, OSError, Exception) as img_err:
        logger.error(Colors.colored(f"Error opening or verifying image {resolved_image_path}: {img_err}", Colors.RED))
        sys.exit(1)


    # --- 2. Extract Metadata ---
    metadata = extract_image_metadata(resolved_image_path)
    print(f"  Date: {Colors.colored(metadata.get('date', 'N/A'), Colors.CYAN)}")
    print(f"  Desc: {Colors.colored(metadata.get('description', 'N/A'), Colors.CYAN)}")
    print(f"  GPS:  {Colors.colored(metadata.get('gps', 'N/A'), Colors.CYAN)}")
    if args.verbose or args.debug:
         exif_data = get_exif_data(resolved_image_path)
         if exif_data:
             pretty_print_exif(exif_data, verbose=True)
         else:
             print("\nNo detailed EXIF data could be extracted.")

    # --- 3. Prepare Prompt ---
    prompt: str
    if args.prompt:
        prompt = args.prompt
        logger.info("Using user-provided prompt.")
    else:
        logger.info("Generating default prompt based on image metadata.")
        prompt_parts = [
            "Provide factual caption, description, keywords/tags for cataloguing/searching.",
            (f"Context: Relates to '{metadata.get('description', '')}'"
             if metadata.get('description') and metadata['description'] != "No description" else ""),
            (f"taken around {metadata.get('date', '')}"
             if metadata.get('date') and metadata['date'] != "Unknown date" else ""),
            (f"near GPS {metadata.get('gps', '')}."
             if metadata.get('gps') and metadata['gps'] != "Unknown location" else ""),
            "Focus on visual content. Avoid repeating context unless visible."
        ]
        prompt = " ".join(filter(None, prompt_parts)).strip()
        logger.debug("Using generated prompt based on metadata.")
    print(f"\n{Colors.colored('--- Using Prompt ---', Colors.CYAN)}\n{prompt}\n{Colors.colored('-'*40, Colors.BLUE)}")

    # --- 4. Find and Process Models ---
    model_identifiers: List[str]
    if args.models:
        model_identifiers = args.models
        logger.info(f"Processing explicitly specified models: {Colors.colored(str(model_identifiers), Colors.GREEN)}")
    else:
        logger.info("Scanning cache for models to process...")
        model_identifiers = get_cached_model_ids()

    results: List[ModelResult] = []
    if not model_identifiers:
        msg = ("\nWarning: No models specified and none found in cache." if not args.models
               else "\nWarning: No models specified via --models argument.")
        print(Colors.colored(msg, Colors.YELLOW), file=sys.stderr)
        if not args.models:
            print("Ensure models are downloaded and cache is accessible.", file=sys.stderr)
    else:
        print(f"\nProcessing {Colors.colored(str(len(model_identifiers)), Colors.GREEN)} model(s)...")
        separator = Colors.colored(f"\n{'=' * 80}\n", Colors.BLUE)
        for model_id in model_identifiers:
            print(separator)
            is_vlm_verbose = args.verbose or args.debug
            result = process_image_with_model(
                model_identifier=model_id, image_path=resolved_image_path,
                prompt=prompt, max_tokens=args.max_tokens,
                verbose=is_vlm_verbose, trust_remote_code=args.trust_remote_code,
                temperature=args.temperature
            )
            results.append(result)
            # Print immediate feedback
            if result.success:
                print(f"\n--- {Colors.colored(f'Output from {model_id.split('/')[-1]}', Colors.GREEN)} ---")
                print(result.output)
            else:
                print(f"--- {Colors.colored(f'Processing failed for model: {model_id}', Colors.RED)} ---")
                # Conditionally print captured output if it exists
                if result.captured_output_on_fail:
                    print(Colors.colored("Captured output during failure:", Colors.YELLOW))
                    # Maybe limit how much is printed to console?
                    capture_snippet = result.captured_output_on_fail[:1000]
                    if len(result.captured_output_on_fail) > 1000:
                        capture_snippet += '...'
                    print(capture_snippet)

    # --- 5. Print Summary Statistics ---
    if results:
        print(Colors.colored(f"\n{'=' * 80}\n", Colors.BLUE)) # Separator
        print_model_stats(results) # Function call
    else:
        print(Colors.colored("\nNo models processed. No performance summary generated.", Colors.YELLOW))

    # --- 6. Generate HTML Report ---
    html_output_path = args.output_html.resolve()
    if results:
        # Pass collected versions to the report generator
        generate_html_report(results, html_output_path, library_versions)
    else:
        logger.info(f"Skipping HTML report generation to {html_output_path} as no models were processed.")

    # --- 7. Print Version Info to Console ---
    # Print versions after all processing and reporting is done
    print_version_info(library_versions)

    # --- Calculate and Print Total Time ---
    overall_time = time.perf_counter() - overall_start_time
    print(f"\nTotal execution time: {Colors.colored(f'{overall_time:.2f} seconds', Colors.GREEN)}.")


if __name__ == "__main__":
    # Setup Argument Parser
    parser: argparse.ArgumentParser = argparse.ArgumentParser(
        description="Analyze image with MLX VLMs.",
        formatter_class=argparse.ArgumentDefaultsHelpFormatter
    )
    # Add arguments (separated for clarity)
    parser.add_argument("-f", "--folder", type=Path, default=DEFAULT_FOLDER, help="Folder to scan.")
    parser.add_argument("--output-html", type=Path, default=DEFAULT_HTML_OUTPUT, help="Output HTML report file.")
    parser.add_argument("--models", nargs='+', type=str, default=None, help="Specify models by ID/path. Overrides cache scan.")
    parser.add_argument("--trust-remote-code", action=argparse.BooleanOptionalAction, default=True, help="Allow custom code from Hub models (SECURITY RISK).")
    parser.add_argument("-p", "--prompt", type=str, default=None, help="Custom VLM prompt.")
    parser.add_argument("-m", "--max-tokens", type=int, default=DEFAULT_MAX_TOKENS, help="Max new tokens to generate.")
    parser.add_argument("-t", "--temperature", type=float, default=DEFAULT_TEMPERATURE, help=f"Sampling temperature (default: {DEFAULT_TEMPERATURE}).")
    parser.add_argument("-v", "--verbose", action="store_true", help="Enable verbose output (INFO logging).")
    parser.add_argument("-d", "--debug", action="store_true", help="Enable debug logging (DEBUG level).")

    # Parse arguments
    parsed_args: argparse.Namespace = parser.parse_args()

    # --- Main Execution ---
    try:
        main(parsed_args)
    except Exception as main_err:
        # Log final unhandled exceptions with color
        logger.exception(Colors.colored(f"An unexpected error occurred during main execution: {main_err}", Colors.RED))
        sys.exit(1) # Exit with error status

Maybe I am not constructing the prompt correctly. The only other diagnostic that I don't understand is

huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)

--- Library Versions ---
Pillow          : 11.2.1
huggingface-hub : 0.30.2
mlx             : 0.24.2
mlx-lm          : 0.22.4
mlx-vlm         : 0.1.22
transformers    : 4.51.2

[Blaizzy]

True

I noticed the same for a couple models

Which ones are the problem for you?

[Blaizzy]

@jrp2014 this solves it:

#296

[jrp2014]

With the smoke test, I am getting:

python  test_smoke.py --models-file models.txt --image ~/Pictures/Processed/20250412-160700_DSC03314.jpg --language-only-prompt "Provide factual caption, description, keywords/tags for cataloguing/searching. taken around 2025-04-13 22:05:21 Focus on visual content. Avoid repeating context unless visible."

  0%|                                                                                            | 0/27 [00:00<?, ?it/s]╭──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ Testing HuggingFaceTB/SmolVLM-Instruct                                                                               │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
Loading model...
Fetching 12 files: 100%|█████████████████████████████████████████████████████████████| 12/12 [00:00<00:00, 14434.08it/s]
Fetching 12 files: 100%|█████████████████████████████████████████████████████████████| 12/12 [00:00<00:00, 26729.50it/s]
✓ Model loaded successfully in 0.69 seconds                                                      | 0/12 [00:00<?, ?it/s]


Testing vision-language generation...
==========
Files: ['/Users/jrp/Pictures/Processed/20250412-160700_DSC03314.jpg'] 

Prompt: <|im_start|>User:<image>Describe this image.<end_of_utterance>
Assistant:
 A narrow street with a brick building on the left and a white building on the right. The street is made of asphalt and has yellow lines marking the center. The buildings are old and have different colors and styles. The brick building on the left has a large window and a door. The white building on the right has a large window and a door.
==========
Prompt: 1560 tokens, 885.675 tokens-per-sec
Generation: 72 tokens, 116.054 tokens-per-sec
Peak memory: 5.735 GB
✓ vision-language generation successful


Testing language-only generation...
==========
Files: None 

Prompt: <|im_start|>User: Provide factual caption, description, keywords/tags for cataloguing/searching. taken around 2025-04-13 22:05:21 Focus on visual content. Avoid repeating context unless visible.<end_of_utterance>
Assistant:
 A man and woman are standing in front of a large, white, rectangular structure. The man is wearing a white shirt and black pants, while the woman is wearing a white dress. The structure is made of white panels and has a large, circular opening in the front. The man and woman are standing in front of the structure, and the woman is looking at the man. The man is looking at the woman. The structure is in a field, and there are trees and grass in the background
==========
Prompt: 57 tokens, 1466.544 tokens-per-sec
Generation: 100 tokens, 124.973 tokens-per-sec
Peak memory: 4.584 GB
✓ language-only generation successful


Cleaning up...
✓ Cleanup complete

The first is right, the second isn't, and I seem to get the second with my own tests. Is there something wrong with the way I am calling the generator (note the Files:None)?

def process_image_with_model(
    model_identifier: str,
    image_path: Path,
    prompt: str,
    max_tokens: int,
    verbose: bool,
    trust_remote_code: bool = True,
    temperature: float = DEFAULT_TEMPERATURE,
) -> ModelResult:
    """Process an image with a Vision Language Model."""
    logger.info(f"Processing '{image_path.name}' with model: {model_identifier}")

    model = tokenizer = None
    arch, gpu_info = get_system_info()

    try:
        validate_temperature(temperature)
        validate_image_accessible(image_path)

        # Log system info in debug mode
        if logger.isEnabledFor(logging.DEBUG):
            logger.debug(f"System: {arch}, GPU: {gpu_info}")

        with timeout(30):  # 30 second timeout
            mx.clear_cache()
            mx.reset_peak_memory()

            initial_mem = mx.get_active_memory() / 1024 / 1024
            initial_cache = mx.get_cache_memory() / 1024 / 1024
            start_time = time.perf_counter()

            model, tokenizer = load(model_identifier, trust_remote_code=trust_remote_code)
            config = load_config(model_identifier, trust_remote_code=trust_remote_code)

            formatted_prompt = apply_chat_template(tokenizer, config, prompt, num_images=1)
            output = generate(
                model=model,
                processor=tokenizer,
                prompt=formatted_prompt,
                image_paths=[str(image_path)],
                max_tokens=max_tokens,
                verbose=verbose,
                temp=temperature
            )

            mx.eval(model.parameters())

            final_stats = MemoryStats(
                active=mx.get_active_memory() / 1024 / 1024 - initial_mem,
                cached=mx.get_cache_memory() / 1024 / 1024 - initial_cache,
                peak=mx.get_peak_memory() / 1024 / 1024,
                time=time.perf_counter() - start_time
            )

            return ModelResult(
                model_name=model_identifier,
                success=True,
                output=str(output) if output is not None else "",
                stats=final_stats
            )

    except TimeoutError:
        logger.error(f"Timeout while processing model {model_identifier}")
        return ModelResult(
            model_name=model_identifier,
            success=False,
            error_stage="timeout",
            error_message="Operation timed out"
        )
    except Exception as e:
        error_stage = "model_load" if model is None else "generate"
        logger.error(f"Failed during {error_stage}: {type(e).__name__}: {e}")
        if logger.isEnabledFor(logging.DEBUG):
            traceback.print_exc()
        return ModelResult(
            model_name=model_identifier,
            success=False,
            error_stage=error_stage,
            error_message=str(e)
        )
    finally:
        del model
        del tokenizer
        mx.clear_cache()
        mx.reset_peak_memory()

2025-04-16 22:47:04,892 - INFO - Processing '20250412-160700_DSC03314.jpg' with model: HuggingFaceTB/SmolVLM-Instruct
2025-04-16 22:47:05,051 - DEBUG - tag: ImageDescription (270) - type: string (2) Tag Location: 18258 - Data Location: 146 - value: <table: 270 bytes>

...

2025-04-16 22:47:05,053 - DEBUG - Starting new HTTPS connection (1): huggingface.co:443
2025-04-16 22:47:05,572 - DEBUG - https://huggingface.co:443 "GET /api/models/HuggingFaceTB/SmolVLM-Instruct/revision/main HTTP/11" 200 5011
Fetching 12 files: 100%|█████████████████████████████████████████████████████████████| 12/12 [00:00<00:00, 47082.93it/s]
2025-04-16 22:47:06,030 - DEBUG - https://huggingface.co:443 "GET /api/models/HuggingFaceTB/SmolVLM-Instruct/revision/main HTTP/11" 200 5011
Fetching 12 files: 100%|█████████████████████████████████████████████████████████████| 12/12 [00:00<00:00, 12589.21it/s]
==========
Files: **None** 

Prompt: <|im_start|>User:<image>Provide factual caption, description, keywords/tags for cataloguing/searching. taken around 2025-04-13 22:05:21 Focus on visual content. Avoid repeating context unless visible.<end_of_utterance>
Assistant:
 A young woman with long, dark hair and a serene expression on her face is standing in front of a large, ornate mirror. The mirror is framed in an ornate gold frame, and it is surrounded by a dark red curtain. The woman is wearing a long, flowing gown that is made of a soft, silky fabric. The gown is sleeveless, and it has a high collar. The woman's hair is pulled back into a loose bun, and she is wearing a simple necklace. The woman is standing in front of the mirror, and she is looking at her reflection. The mirror is reflecting her face, and she is looking at her reflection with a serene expression on her face. The woman's eyes are soft and calm, and she is not smiling. The mirror is reflecting her face, and she is looking at her reflection with a serene expression on her face. The woman's eyes are soft and calm, and she is not smiling. The mirror is reflecting her face, and she is looking at her reflection with a serene expression on her face. The woman's eyes are soft and calm, and she is not smiling.
==========
Prompt: 58 tokens, 1152.625 tokens-per-sec
Generation: 232 tokens, 124.296 tokens-per-sec
Peak memory: 4.638 GB

--- Library Versions ---
Pillow          : 11.2.1
huggingface-hub : 0.30.2
mlx             : 0.24.2.dev20250416+99eefd2e
mlx-lm          : 0.22.4
mlx-vlm         : 0.1.23
transformers    : 4.51.3

[Blaizzy]

The second one is language only. There is no image or image tag passed

[Blaizzy]

The smoke test file, first check the full VLM then tests the Language module only.

If you want to test VLM on different prompts you need to alter it