Attendance Module: Support for fallback mechanisms and missing attendances.

[mubashir-rehman]

Issues:

1. If for some reason Biometric attendance is missed for some employees. The one way is to reset the biometric last fetch date to a previous one. This creates new records for missed attendances. However, the issue comes when there are missing check outs in the records. There is a possibility that a check out from date previous than check in is assigned in attendance activity as well as work records. Leading to negative work hours on a given day. Sometimes, it can go other way around as well with more than 24 hours of work time on a particular day.
2. No fallback mechanism for connection lost with Biometric devices. We use ZKTeco devices and are relying on ZKBioTime web api. I can share the code (It is a mess because of time constraints)
3. pyzk uses ping to test for connection. For security reasons, we have disabled outside network pings (HRMS deployed on cloud, device in office). We switched to ZKBio Time web api.
   I can share the code as well as detailed logs with screenshots if needed.

[mubashir-rehman]

"""
Module for managing biometric devices and employee attendance.

Includes classes and functions for adding, editing, and deleting biometric devices,
as well as scheduling attendance capture. Also provides views for managing employees,
registered on biometric devices.
"""

import json
import logging
from datetime import datetime, timedelta
from threading import Event, Thread
from urllib.parse import parse_qs, unquote
from time import sleep
import requests

import pytz
from apscheduler.schedulers.background import BackgroundScheduler
from django.contrib import messages
from django.http import HttpResponse, JsonResponse
from django.shortcuts import redirect, render
from django.template.loader import render_to_string
from django.utils import timezone as django_timezone
from django.utils.translation import gettext as __
from django.utils.translation import gettext_lazy as _
from zk import ZK
from zk import exception as zk_exception

from attendance.methods.utils import Request
from attendance.models import AttendanceActivity
from attendance.views.clock_in_out import clock_in, clock_out
from base.methods import get_key_instances, get_pagination
from employee.models import Employee, EmployeeWorkInformation
from horilla.decorators import (
    hx_request_required,
    install_required,
    login_required,
    permission_required,
)
from horilla.filters import HorillaPaginator
from horilla.horilla_settings import BIO_DEVICE_THREADS
from horilla.settings import TIME_ZONE

from .anviz import CrossChexCloudAPI
from .cosec import COSECBiometric
from .dahua import DahuaAPI
from .etimeoffice import ETimeOfficeAPI
from .filters import BiometricDeviceFilter
from .forms import (
    BiometricDeviceForm,
    BiometricDeviceSchedulerForm,
    CosecUserAddForm,
    COSECUserForm,
    DahuaUserForm,
    EmployeeBiometricAddForm,
    MapBioUsers,
)
from .models import BiometricDevices, BiometricEmployees, COSECAttendanceArguments

logger = logging.getLogger(__name__)

# ZKBiotime API Helper Functions
def get_zkbiotime_token(api_url, username, password):
    """
    Get authentication token from ZKBiotime Web API
    """
    try:
        token_url = f"{api_url}/api-token-auth/"
        data = {"username": username, "password": password}
        headers = {"Content-Type": "application/json"}
        
        logger.info(f"Requesting ZKBiotime token from {token_url}")
        response = requests.post(
            token_url, 
            data=json.dumps(data), 
            headers=headers, 
            timeout=10
        )
        
        if response.status_code == 200:
            token_data = response.json()
            token = token_data.get("token")
            logger.info("ZKBiotime authentication token received successfully")
            return token
        else:
            logger.error(f"ZKBiotime auth failed: {response.status_code} - {response.text}")
            return None
            
    except Exception as e:
        logger.error(f"Error getting ZKBiotime token: {e}")
        return None

def test_zkbiotime_connection(api_url, username, password):
    """
    Test connection to ZKBiotime Web API
    """
    try:
        token = get_zkbiotime_token(api_url, username, password)
        if not token:
            return False
            
        # Test with a simple API call
        headers = {
            "Authorization": f"Token {token}",
            "Content-Type": "application/json"
        }
        
        test_url = f"{api_url}/personnel/api/employees/"
        response = requests.get(test_url, headers=headers, params={"page_size": 1}, timeout=10)
        
        success = response.status_code == 200
        logger.info(f"ZKBiotime connection test {'successful' if success else 'failed'}")
        return success
        
    except Exception as e:
        logger.error(f"ZKBiotime connection test failed: {e}")
        return False

def get_zkbiotime_transactions(api_url, token, start_time=None, page_size=50):
    """
    Get attendance transactions from ZKBiotime Web API
    """
    try:
        headers = {
            "Authorization": f"Token {token}",
            "Content-Type": "application/json"
        }
        
        params = {
            "format": "json",
            "page": 1,
            "page_size": page_size
        }
        
        # Add time filter if provided
        if start_time:
            params["start_time"] = start_time.strftime('%Y-%m-%d %H:%M:%S')
        
        transactions_url = f"{api_url}/iclock/api/transactions/"
        logger.debug(f"Fetching transactions from {transactions_url}")
        
        response = requests.get(transactions_url, headers=headers, params=params, timeout=30)
        
        if response.status_code == 200:
            data = response.json()
            
            # Handle different response formats
            transactions = []
            if "data" in data:
                transactions = data["data"]
            elif "results" in data:
                transactions = data["results"]
            elif isinstance(data, list):
                transactions = data
            
            logger.info(f"Retrieved {len(transactions)} transactions from ZKBiotime API")
            return transactions
        else:
            logger.error(f"ZKBiotime API request failed: {response.status_code} - {response.text}")
            return []
            
    except Exception as e:
        logger.error(f"Error fetching ZKBiotime transactions: {e}")
        return []

def process_zkbiotime_transaction(transaction, device):
    """
    Process a single ZKBiotime transaction and convert to attendance record.
    Fixed timezone handling to prevent clock-out date issues.
    """
    try:
        # Extract transaction data
        emp_code = transaction.get('emp_code')
        punch_time_str = transaction.get('punch_time')
        punch_state = transaction.get('punch_state')  # '0' for check-in, '1' for check-out
        
        if not emp_code or not punch_time_str or punch_state is None:
            logger.warning(f"Incomplete transaction data: {transaction}")
            return None
        
        # Parse punch time with improved timezone handling
        try:
            # Try different datetime formats
            punch_time = None
            for fmt in ['%Y-%m-%d %H:%M:%S', '%Y-%m-%dT%H:%M:%S', '%Y-%m-%dT%H:%M:%S%z']:
                try:
                    punch_time = datetime.strptime(punch_time_str, fmt)
                    break
                except ValueError:
                    continue
            
            if not punch_time:
                raise ValueError(f"Unable to parse datetime: {punch_time_str}")
                
            # FIXED: Proper timezone handling
            user_tz = pytz.timezone(TIME_ZONE)
            
            if punch_time.tzinfo is None:
                # If naive datetime, assume it's in device timezone
                punch_time_aware = user_tz.localize(punch_time)
            else:
                # If already timezone-aware, convert to user timezone
                punch_time_aware = punch_time.astimezone(user_tz)
            
            logger.debug(f"Processed punch time: {punch_time_str} -> {punch_time_aware}")
            
        except Exception as time_error:
            logger.error(f"Could not parse punch time '{punch_time_str}': {time_error}")
            return None
        
        # Find employee mapping
        bio_employee = BiometricEmployees.objects.filter(
            user_id=emp_code, device_id=device
        ).first()
        
        if not bio_employee:
            logger.debug(f"No BiometricEmployees record for emp_code {emp_code} on device {device.id}")
            return None
        
        # Validate employee has user account
        if not bio_employee.employee_id.employee_user_id:
            logger.warning(f"Employee {bio_employee.employee_id} has no user account")
            return None
        
        # FIXED: Additional validation for reasonable timestamps
        now = django_timezone.now()
        
        # Skip transactions older than 30 days (to avoid processing old data)
        if punch_time_aware < now - timedelta(days=30):
            logger.debug(f"Skipping old transaction from {punch_time_aware} (more than 30 days old)")
            return None
        
        # Skip future transactions (more than 1 hour ahead)
        if punch_time_aware > now + timedelta(hours=1):
            logger.debug(f"Skipping future transaction from {punch_time_aware}")
            return None
        
        return {
            'bio_employee': bio_employee,
            'datetime': punch_time_aware,
            'punch_state': str(punch_state),
            'punch_code': 0 if str(punch_state) == '0' else 1,
            'user_id': emp_code,
            'employee_user_id': bio_employee.employee_id.employee_user_id
        }
        
    except Exception as e:
        logger.error(f"Error processing ZKBiotime transaction: {e}")
        return None

def str_time_seconds(time):
    """
    this method is used reconvert time in H:M formate string back to seconds and return it
    args:
        time : time in H:M format
    """

    ftr = [3600, 60, 1]
    return sum(a * b for a, b in zip(ftr, map(int, time.split(":"))))


def paginator_qry(qryset, page_number):
    """
    This method is used to paginate query set
    """
    paginator = HorillaPaginator(qryset, get_pagination())
    qryset = paginator.get_page(page_number)
    return qryset


def biometric_paginator_qry(data_list, page_number, per_page=25):
    """
    This function is used to paginate a list of dictionaries.
    """
    start_index = (page_number - 1) * per_page
    end_index = page_number * per_page
    paginated_data = {}
    paginated_data["users"] = data_list[start_index:end_index]

    total_items = len(data_list)
    total_pages = (total_items + per_page - 1) // per_page
    has_previous = page_number > 1
    has_next = page_number < total_pages

    paginated_data["paginator"] = {
        "number": page_number,
        "previous_page_number": page_number - 1 if has_previous else None,
        "next_page_number": page_number + 1 if has_next else None,
        "num_pages": total_pages,
        "has_previous": has_previous,
        "has_next": has_next,
    }
    return paginated_data


def biometric_set_time(conn):
    """
    Sets the time on the biometric device using the provided connection.

    Parameters:
    - conn: The connection to the biometric device.

    Returns:
    None
    """
    new_time = datetime.today()
    conn.set_time(new_time)


class ZKBioAttendance(Thread):
    """
    Represents a thread for capturing live attendance data from a ZKTeco biometric device.
    Now includes ZKBiotime Web API fallback mechanism.
    """

    def __init__(self, machine_ip, port_no, password):
        """
        Initialize ZKBioAttendance thread
        
        Args:
            machine_ip (str): IP address of the ZK device
            port_no (int): Port number of the ZK device
            password (str/int): Device password
        """
        super().__init__()
        self.machine_ip = machine_ip
        self.port_no = port_no
        self.password = int(password)
        self._stop_event = Event()
        self.conn = None
        
        # ZKBiotime API fallback configuration
        self.zkbiotime_api_url = ""
        self.zkbiotime_username = ""
        self.zkbiotime_password = ""
        self.zkbiotime_token = None
        self.zkbiotime_token_expiry = None
        self.api_fallback_active = True

        # Add pagination tracking
        self.last_page = 1
        self.page_size = 100
        self.processing_lock = Event()  # Lock for request throttling
        self.min_request_interval = 5  # Minimum seconds between API requests

        # Add flag to track if we've switched to API mode permanently
        self.api_mode_permanent = False
        
        
        # Get device for last fetch date
        try:
            self.device = BiometricDevices.objects.filter(
                machine_ip=machine_ip,
                port=port_no
            ).first()
            self.last_api_fetch = self.device.last_fetch_date if self.device else None
            logger.debug(f"{machine_ip}:{port_no} - Last API fetch date: {self.last_api_fetch}")
        except Exception as e:
            logger.error(f"Error getting device info: {e}")
            self.device = None
            self.last_api_fetch = None
            
        logger.debug(f"Initialized ZKBioAttendance for {machine_ip}:{port_no}")
        
    def get_valid_zkbiotime_token(self):
        """
        Get a valid ZKBiotime token, refreshing if necessary
        """
        try:
            # Check if current token is still valid
            if (self.zkbiotime_token and self.zkbiotime_token_expiry and 
                datetime.now() < self.zkbiotime_token_expiry):
                return self.zkbiotime_token
            
            # Get new token
            logger.info("Refreshing ZKBiotime API token")
            token = get_zkbiotime_token(
                self.zkbiotime_api_url,
                self.zkbiotime_username,
                self.zkbiotime_password
            )
            
            if token:
                self.zkbiotime_token = token
                self.zkbiotime_token_expiry = datetime.now() + timedelta(hours=12)
                logger.info("ZKBiotime token refreshed successfully")
                return token
            else:
                logger.error("Failed to refresh ZKBiotime token")
                return None
                
        except Exception as e:
            logger.error(f"Error getting valid ZKBiotime token: {e}")
            return None

    def try_zkbiotime_fallback(self, device):
        """
        Attempt to switch to ZKBiotime API fallback
        """
        try:
            logger.info("Attempting ZKBiotime API fallback for device %s:%s", 
                       self.machine_ip, self.port_no)
            
            # Test connection first
            if test_zkbiotime_connection(
                self.zkbiotime_api_url,
                self.zkbiotime_username,
                self.zkbiotime_password
            ):
                logger.info("ZKBiotime API fallback connection successful")
                self.api_fallback_active = True
                self.api_mode_permanent = True  # Mark API mode as permanent
                self.last_api_fetch = datetime.now()
                return True
            else:
                logger.error("ZKBiotime API fallback connection failed")
                return False
                
        except Exception as e:
            logger.error(f"Error in ZKBiotime fallback attempt: {e}")
            return False

    def fetch_api_attendance(self, device):
        """
        Fetch attendance data from ZKBiotime API with improved pagination handling
        """
        try:
            # Don't make new request if still processing previous data
            if self.processing_lock.is_set():
                logger.debug("Still processing previous data, skipping new fetch")
                return []

            token = self.get_valid_zkbiotime_token()
            if not token:
                logger.error("No valid ZKBiotime token available")
                return []

            # Set processing lock
            self.processing_lock.set()

            try:
                # Determine start time for fetching - FIXED
                if device.last_fetch_date and device.last_fetch_time:
                    # Combine date and time properly
                    start_time = datetime.combine(device.last_fetch_date, device.last_fetch_time)
                elif device.last_fetch_date:
                    # Only date available, start from beginning of that day
                    start_time = datetime.combine(device.last_fetch_date, datetime.min.time())
                else:
                    # First run - get data from a reasonable starting point
                    start_time = datetime.now() - timedelta(days=7)  # Last 7 days

                logger.debug(f"Fetching API transactions since {start_time}")

                # Initialize variables for pagination
                all_transactions = []
                next_page_url = None
                current_page = 1
                latest_punch_time = None  # Track latest transaction time

                while True:
                    # Fetch current page of transactions
                    headers = {
                        "Authorization": f"Token {token}",
                        "Content-Type": "application/json"
                    }

                    # For first request use base URL, for subsequent requests use next_page_url
                    if next_page_url:
                        response = requests.get(next_page_url, headers=headers, timeout=10)
                    else:
                        params = {
                            "page": current_page,
                            "start_time": start_time.strftime('%Y-%m-%dT%H:%M:%S')
                        }

                        response = requests.get(
                            f"{self.zkbiotime_api_url}/iclock/api/transactions/",
                            headers=headers,
                            params=params,
                            timeout=10
                        )

                    if response.status_code == 200:
                        data = response.json()
                        transactions = data.get("data", [])
                        next_page_url = data.get("next")  # Get URL for next page
                        total_count = data.get("count", 0)

                        logger.info(f"Fetched {len(transactions)} transactions from page {current_page}")
                        logger.debug(f"Total records available: {total_count}")

                        # Process transactions from current page
                        processed_transactions = []
                        for transaction in transactions:
                            processed = process_zkbiotime_transaction(transaction, device)
                            if processed:
                                processed_transactions.append(processed)

                                # Track the latest punch time
                                if not latest_punch_time or processed['datetime'] > latest_punch_time:
                                    latest_punch_time = processed['datetime']

                        # Add processed transactions to main list
                        all_transactions.extend(processed_transactions)

                        # Break if no more pages
                        if not next_page_url:
                            logger.info("No more pages to fetch")
                            break

                        # Increment page counter
                        current_page += 1

                        # Optional: Add delay between requests to avoid overwhelming the API
                        sleep(1)

                    else:
                        logger.error(f"API request failed: {response.status_code}")
                        break

                logger.info(f"Total processed records across all pages: {len(all_transactions)}")

                # Update last fetch time ONLY if we got new transactions - FIXED
                if all_transactions and latest_punch_time:
                    device.last_fetch_date = latest_punch_time.date()
                    device.last_fetch_time = latest_punch_time.time()
                    device.save()
                    logger.debug(f"Updated device last fetch date to {device.last_fetch_date} {device.last_fetch_time}")

                return all_transactions

            finally:
                # Release processing lock and enforce minimum interval
                self.processing_lock.clear()
                sleep(self.min_request_interval)

        except Exception as e:
            logger.error(f"Error fetching API attendance: {e}")
            self.processing_lock.clear()  # Ensure lock is released on error
            return []

    def try_restore_direct_connection(self):
        """
        Check if we should attempt to restore direct ZK connection
        """
        # If API mode is permanent, don't try to restore
        if self.api_mode_permanent:
            logger.debug("API mode is permanent, skipping direct connection restore attempt")
            return False
            
        try:
            logger.debug("Attempting to restore direct ZK connection")
            zk_device = ZK(
                self.machine_ip,
                port=self.port_no,
                timeout=5,
                password=self.password,
                force_udp=False,
                ommit_ping=False,
            )
            test_conn = zk_device.connect()
            
            if test_conn:
                # Test if connection is actually working
                test_conn.test_voice(12)
                test_conn.disconnect()
                logger.info("Direct ZK connection restored successfully")
                return True
            else:
                logger.debug("Direct ZK connection still not available")
                return False
                
        except Exception as e:
            logger.debug(f"Direct connection restore attempt failed: {e}")
            return False

    def run(self):
        logger.info("Thread started for device %s:%s with API fallback support", 
                   self.machine_ip, self.port_no)
        try:
            try:
                if not self.api_mode_permanent:  # Only try direct if not in permanent API mode
                    logger.debug("Attempting to connect to ZK device at %s:%s", 
                               self.machine_ip, self.port_no)
                    zk_device = ZK(
                        self.machine_ip,
                        port=self.port_no,
                        timeout=5,
                        password=self.password,
                        force_udp=False,
                        ommit_ping=False,
                    )
                    conn = zk_device.connect()
                    self.conn = conn

                    if not conn:
                        logger.error("Failed to connect to device %s:%s, trying API fallback", 
                                   self.machine_ip, self.port_no)
                        raise Exception("Connection failed")
                else:
                    logger.info("Device is in permanent API mode, skipping direct connection attempt")
                    raise Exception("Permanent API mode")

            except:
                logger.error("Initial connection failed, switching to API fallback")
                self.api_fallback_active = True
                self.api_mode_permanent = True  # Mark as permanent on initial failure
                
                # Get device object for fallback
                device = BiometricDevices.objects.filter(
                    machine_ip=self.machine_ip, port=self.port_no
                ).first()
                
                if device and self.try_zkbiotime_fallback(device):
                    logger.info("Successfully switched to ZKBiotime API fallback")
                    # Continue with API-only mode
                    self.run_api_only_mode(device)
                return

            logger.info("Successfully connected to device %s:%s", self.machine_ip, self.port_no)
            device = BiometricDevices.objects.filter(
                machine_ip=self.machine_ip, port=self.port_no
            ).first()

            if not device:
                logger.warning("No BiometricDevices entry found for %s:%s", self.machine_ip, self.port_no)
                return

            if not device.is_live:
                logger.info("Device %s:%s is not set to live capture", self.machine_ip, self.port_no)
                return

            connection_lost = False
            api_restore_attempt_counter = 0

            while not self._stop_event.is_set():
                try:
                    if not self.api_fallback_active:
                        # Normal direct ZK mode
                        logger.debug("Operating in direct ZK mode")
                        try:
                            attendances = conn.live_capture()
                            logger.debug("Successfully captured live attendance from direct connection")
                            connection_lost = False
                        except Exception as e:
                            logger.warning("Direct ZK live_capture failed: %s", e)
                            connection_lost = True
                            
                            # Try to switch to API fallback
                            if self.try_zkbiotime_fallback(device):
                                logger.info("Switched to ZKBiotime API fallback mode")
                                continue
                            else:
                                logger.error("Both direct connection and API fallback failed")
                                break
                    else:
                        # API fallback mode
                        logger.debug("Operating in API fallback mode")
                        try:
                            # Fetch from API
                            api_attendance = self.fetch_api_attendance(device)
                            
                            # Convert API format to direct format for consistent processing
                            attendances = []
                            for record in api_attendance:
                                # Create a mock attendance object similar to ZK format
                                mock_attendance = type('APIAttendance', (), {
                                    'user_id': record['user_id'],
                                    'punch': record['punch_code'],
                                    'timestamp': record['datetime'].replace(tzinfo=None)
                                })()
                                attendances.append(mock_attendance)
                            
                            logger.debug(f"Converted {len(attendances)} API records to ZK format")
                            
                            # Periodically try to restore direct connection
                            # api_restore_attempt_counter += 1
                            # if api_restore_attempt_counter >= 12:  # Try every ~60 seconds (5s * 12)
                            #     api_restore_attempt_counter = 0
                            #     if self.try_restore_direct_connection():
                            #         logger.info("Restored direct connection, switching back from API fallback")
                            #         self.api_fallback_active = False
                            #         # Reconnect
                            #         try:
                            #             conn = zk_device.connect()
                            #             self.conn = conn
                            #             if conn:
                            #                 logger.info("Direct connection restored successfully")
                            #                 continue
                            #         except Exception as reconnect_error:
                            #             logger.error(f"Failed to reconnect after restore: {reconnect_error}")
                            #             self.api_fallback_active = True
                            
                        except Exception as api_error:
                            logger.error(f"API fallback error: {api_error}")
                            # Try to restore direct connection on API failure
                            if self.try_restore_direct_connection():
                                logger.info("Switching back to direct connection due to API failure")
                                self.api_fallback_active = False
                                try:
                                    conn = zk_device.connect()
                                    self.conn = conn
                                    continue
                                except Exception as reconnect_error:
                                    logger.error(f"Failed to reconnect: {reconnect_error}")
                                    sleep(5)
                                    continue
                            else:
                                logger.error("Both API and direct connection failed")
                                sleep(5)
                                continue

                    # Process attendance records (same logic for both direct and API)
                    for attendance in attendances or []:
                        if not attendance:
                            # Only do connection testing in direct mode
                            if not self.api_fallback_active:
                                is_alive = True
                                try:
                                    is_alive = conn.test_voice(12)
                                except Exception as e:
                                    logger.warning("test_voice() raised %s, marking connection lost", e)
                                    is_alive = False
                                    
                                if not is_alive:
                                    if not connection_lost:
                                        connection_lost = True
                                        logger.error("🔴 Connection lost to device %s:%s, attempting API fallback",
                                                   self.machine_ip, self.port_no)
                                        
                                        # Try API fallback
                                        if self.try_zkbiotime_fallback(device):
                                            logger.info("Switched to API fallback due to connection loss")
                                            break  # Break from attendance loop to switch modes
                                    
                                    # retry loop until reconnected or API fallback activated
                                    while connection_lost and not self._stop_event.is_set() and not self.api_fallback_active:
                                        sleep(5)
                                        try:
                                            if conn.test_voice(12):
                                                is_alive = True
                                                connection_lost = False
                                                logger.info("🟢 Connection reestablished with device %s:%s",
                                                           self.machine_ip, self.port_no)
                                                conn.disconnect()
                                                logger.info("Disconnecting before making another connection.")

                                                # fetch logs before resuming
                                                try:
                                                    logger.info(f"Fetching ZK attendance logs for device {device.id}")
                                                    attendance_count, error_msg = zk_biometric_attendance_logs(device)
                                                    if isinstance(attendance_count, int):
                                                        logger.info(f"Fetched {attendance_count} attendance records before live start.")
                                                        conn = zk_device.connect()
                                                        self.conn = conn
                                                        logger.info("Logs fetched. Resuming live capture.")
                                                    else:
                                                        logger.warning(f"Could not fetch logs before live start: {error_msg}")
                                                except Exception as e:
                                                    logger.exception(f"Error fetching ZK logs for device {device.id}: {e}")
                                        except Exception as e:
                                            # Try API fallback if direct connection keeps failing
                                            if self.try_zkbiotime_fallback(device):
                                                logger.info("Switched to API fallback during retry loop")
                                                break
                                            logger.debug("Retry test_voice() exception: %s", e)
                            continue

                        try:
                            user_id = attendance.user_id
                            punch_code = attendance.punch
                            date_time = django_timezone.make_aware(attendance.timestamp)
                            date = date_time.date()
                            time = date_time.time()

                            device.last_fetch_date = date
                            device.last_fetch_time = time
                            logger.debug(f"Live capture state for {device.id} before save in ZKBioAttendance: {BiometricDevices.objects.get(id=device.id).is_live}")

                            device.save()
                            logger.debug(f"Live capture state for {device.id} after save in ZKBioAttendance: {BiometricDevices.objects.get(id=device.id).is_live}")

                            bio_id = BiometricEmployees.objects.filter(
                                user_id=user_id, device_id=device
                            ).first()

                            if not bio_id:
                                logger.debug("No BiometricEmployees record for user_id %s on device %s", user_id, device)
                                continue

                            req = Request(
                                user=bio_id.employee_id.employee_user_id,
                                date=date,
                                time=time,
                                datetime=date_time,
                            )

                            if punch_code in {0, 3, 4}:
                                logger.info("Processing clock-in for user %s at %s (via %s)", 
                                          req.user, date_time, 
                                          "API" if self.api_fallback_active else "direct")
                                try:
                                    clock_in(req)
                                except Exception as e:
                                    logger.exception("Error in clock_in for user %s: %s", req.user, e)
                            else:
                                logger.info("Processing clock-out for user %s at %s (via %s)", 
                                          req.user, date_time,
                                          "API" if self.api_fallback_active else "direct")
                                try:
                                    clock_out(req)
                                except Exception as e:
                                    logger.exception("Error in clock_out for user %s: %s", req.user, e)

                        except Exception as e:
                            logger.exception("Unexpected error processing attendance record: %s", e)
                            continue

                    # Sleep different intervals based on mode
                    if self.api_fallback_active:
                        sleep(5)  # API mode - less frequent polling

                except Exception as loop_error:
                    logger.error("Error in main attendance processing loop: %s", loop_error)
                    sleep(5)
                    continue

        except ConnectionResetError as error:
            logger.error("ConnectionResetError: %s. Attempting restart with API fallback.", error)
            # Try API fallback before restarting thread
            device = BiometricDevices.objects.filter(
                machine_ip=self.machine_ip, port=self.port_no
            ).first()
            
            if device and self.try_zkbiotime_fallback(device):
                logger.info("Switched to API fallback after ConnectionResetError")
                self.run_api_only_mode(device)
            else:
                # restart thread as last resort
                ZKBioAttendance(self.machine_ip, self.port_no, self.password).start()
        except Exception as e:
            logger.exception("Unhandled exception in ZKBioAttendance run(): %s", e)
        finally:
            if self.conn:
                try:
                    self.conn.end_live_capture = True
                    logger.info("Ended live capture for device %s:%s", self.machine_ip, self.port_no)
                except Exception as e:
                    logger.exception("Error stopping live capture: %s", e)
            logger.info("Thread exiting for device %s:%s (API fallback: %s)", 
                       self.machine_ip, self.port_no, self.api_fallback_active)

    def run_api_only_mode(self, device):
        """
        Run in API-only mode with improved punch state handling - FIXED
        """
        logger.info("Running in API-only mode for device %s:%s", self.machine_ip, self.port_no)
    
        while not self._stop_event.is_set():
            try:
                # Fetch attendance from API with pagination
                api_attendance = self.fetch_api_attendance(device)
    
                if api_attendance:
                    # Process each attendance record DIRECTLY - don't convert to mock format
                    for record in api_attendance:
                        try:
                            # Create Request object with proper datetime handling
                            req = Request(
                                user=record['bio_employee'].employee_id.employee_user_id,
                                date=record['datetime'].date(),
                                time=record['datetime'].time(),
                                datetime=record['datetime'],  # This should be timezone-aware
                            )
    
                            # Use punch_state to determine clock in/out
                            if record['punch_state'] == '0':  # Check In
                                logger.info("Processing API clock-in for user %s at %s", 
                                          req.user, record['datetime'])
                                try:
                                    clock_in(req)
                                except Exception as e:
                                    logger.exception("Error in API clock_in for user %s: %s", req.user, e)
                            elif record['punch_state'] == '1':  # Check Out
                                logger.info("Processing API clock-out for user %s at %s", 
                                          req.user, record['datetime'])
                                try:
                                    clock_out(req)
                                except Exception as e:
                                    logger.exception("Error in API clock_out for user %s: %s", req.user, e)
                            else:
                                logger.warning("Unknown punch_state '%s' for user %s", 
                                             record['punch_state'], req.user)
    
                        except Exception as e:
                            logger.exception("Error processing API attendance record: %s", e)
                            continue
                        
                sleep(self.min_request_interval)
    
            except Exception as e:
                logger.error("Error in API-only mode: %s", e)
                sleep(self.min_request_interval)
                continue
            
        logger.info("Exiting API-only mode for device %s:%s", self.machine_ip, self.port_no)

    def stop(self):
        """To stop the ZK live capture mode"""
        logger.info("Stop requested for device %s:%s (API fallback: %s)", 
                   self.machine_ip, self.port_no, self.api_fallback_active)
        if self.conn:
            try:
                self.conn.end_live_capture = True
                device = BiometricDevices.objects.filter(
                machine_ip=self.machine_ip, port=self.port_no
                ).first()
                device.is_live = False
                device.save()
                logger.debug("Set end_live_capture flag on connection for %s:%s", self.machine_ip, self.port_no)
            except Exception as e:
                logger.exception("Error in stop() when ending live capture: %s", e)
        self._stop_event.set()
        logger.info("Stop event set for device %s:%s", self.machine_ip, self.port_no)

class COSECBioAttendanceThread(Thread):
    """
    A thread class that handles the real-time retrieval and processing of
    biometric attendance data from a COSEC biometric device.

    Attributes:
        device_id (int): The ID of the biometric device to interact with.
        _stop_event (threading.Event): An event to signal when to stop the thread.

    Methods:
        run():
            Continuously fetches attendance data from the COSEC device, processes
            it, and updates the last fetched sequence and rollover count.

        stop():
            Signals the thread to stop by setting the _stop_event.
    """

    def __init__(self, device_id):
        super().__init__()
        self.device_id = device_id
        self._stop_event = Event()

    def run(self):
        try:
            device = BiometricDevices.objects.get(id=self.device_id)
            if not device.is_live:
                return

            device_args = COSECAttendanceArguments.objects.filter(
                device_id=device
            ).first()
            last_fetch_roll_ovr_count = (
                int(device_args.last_fetch_roll_ovr_count) if device_args else 0
            )
            last_fetch_seq_number = (
                int(device_args.last_fetch_seq_number) if device_args else 1
            )

            cosec = COSECBiometric(
                device.machine_ip,
                device.port,
                device.bio_username,
                device.bio_password,
                timeout=10,
            )
            while not self._stop_event.is_set():
                attendances = cosec.get_attendance_events(
                    last_fetch_roll_ovr_count, int(last_fetch_seq_number) + 1
                )
                if not isinstance(attendances, list):
                    self._stop_event.wait(5)
                    continue

                for attendance in attendances:
                    ref_user_id = attendance["detail-1"]
                    employee = BiometricEmployees.objects.filter(
                        ref_user_id=ref_user_id
                    ).first()
                    if not employee:
                        continue

                    date_str = attendance["date"]
                    time_str = attendance["time"]
                    attendance_date = datetime.strptime(date_str, "%d/%m/%Y").date()
                    attendance_time = datetime.strptime(time_str, "%H:%M:%S").time()
                    attendance_datetime = datetime.combine(
                        attendance_date, attendance_time
                    )
                    punch_code = attendance["detail-2"]

                    request_data = Request(
                        user=employee.employee_id.employee_user_id,
                        date=attendance_date,
                        time=attendance_time,
                        datetime=django_timezone.make_aware(attendance_datetime),
                    )
                    try:
                        if punch_code in ["1", "3", "5", "7", "9", "0"]:
                            clock_in(request_data)
                        elif punch_code in ["2", "4", "6", "8", "10"]:
                            clock_out(request_data)
                    except Exception as error:
                        logger.error("Error processing attendance: ", error)

                if attendances:
                    last_attendance = attendances[-1]
                    last_fetch_seq_number = last_attendance["seq-No"]
                    last_fetch_roll_ovr_count = last_attendance["roll-over-count"]
                    COSECAttendanceArguments.objects.update_or_create(
                        device_id=device,
                        defaults={
                            "last_fetch_roll_ovr_count": last_fetch_roll_ovr_count,
                            "last_fetch_seq_number": last_fetch_seq_number,
                        },
                    )
                # Sleep to prevent overwhelming the device with requests
                self._stop_event.wait(2)

        except Exception as error:
            device = BiometricDevices.objects.get(id=self.device_id)
            device.is_live = False
            device.save()
            logger.error("Error in COSECBioAttendanceThread: ", error)

    def stop(self):
        """Set the stop event to signal the thread to stop gracefully."""
        self._stop_event.set()


@login_required
@install_required
@permission_required("biometric.view_biometricdevices")
def biometric_devices_view(request):
    """
    Renders a page displaying a list of active biometric devices.

    Parameters:
    - request: The HTTP request object.

    Returns:
    - HttpResponse: The rendered HTML page displaying the list of biometric devices.

    Template:
    - "biometric/view_biometric_devices.html"

    Context:
    - biometric_form (BiometricDeviceForm): Form for adding new biometric devices.
    - devices (QuerySet): Queryset of active biometric devices, ordered by creation date.
    - f (BiometricDeviceFilter): Form for filtering biometric devices.

    """
    biometric_form = BiometricDeviceForm()
    filter_form = BiometricDeviceFilter()
    biometric_devices = BiometricDevices.objects.filter(is_active=True).order_by(
        "-created_at"
    )
    biometric_devices = paginator_qry(biometric_devices, request.GET.get("page"))
    template = "biometric/view_biometric_devices.html"
    context = {
        "biometric_form": biometric_form,
        "devices": biometric_devices,
        "f": filter_form,
    }
    return render(request, template, context)


@login_required
@install_required
@permission_required("biometric.change_biometricdevices")
def biometric_device_schedule(request, device_id):
    """
    Handles scheduling of attendance capture from a biometric device.
    """
    try:
        logger.debug(f"Attempting to fetch device with ID: {device_id}")
        print(f"Attempting to fetch device with ID: {device_id}")
        device = BiometricDevices.objects.get(id=device_id)
    except Exception as e:
        logger.error(f"Failed to retrieve Biometric Device: {e}")
        print(f"Failed to retrieve Biometric Device: {e}")
        return HttpResponse("Device not found", status=404)

    initial_data = {"scheduler_duration": device.scheduler_duration}
    scheduler_form = BiometricDeviceSchedulerForm(initial=initial_data)
    context = {
        "scheduler_form": scheduler_form,
        "device_id": device_id,
    }

    if request.method == "POST":
        logger.debug(f"Received POST request for device ID: {device_id}")
        print(f"Received POST request for device ID: {device_id}")

        scheduler_form = BiometricDeviceSchedulerForm(request.POST)
        if scheduler_form.is_valid():
            logger.info(f"Scheduler form is valid for device {device_id}")
            print(f"Scheduler form is valid for device {device_id}")

            if device.machine_type == "zk":
                try:
                    logger.debug("Processing ZK device scheduling")
                    print("Processing ZK device scheduling")

                    port_no = device.port
                    machine_ip = device.machine_ip
                    password = device.zk_password
                    conn = None

                    zk_device = ZK(
                        machine_ip,
                        port=port_no,
                        timeout=5,
                        password=int(password),
                        force_udp=False,
                        ommit_ping=False,
                    )

                    conn = zk_device.connect()
                    conn.test_voice(index=0)

                    duration = request.POST.get("scheduler_duration")
                    device = BiometricDevices.objects.get(id=device_id)
                    device.scheduler_duration = duration
                    device.is_scheduler = True
                    device.is_live = False
                    device.save()

                    scheduler = BackgroundScheduler()
                    scheduler.add_job(
                        lambda: zk_biometric_attendance_scheduler(device.id),
                        "interval",
                        seconds=str_time_seconds(device.scheduler_duration),
                    )
                    scheduler.start()

                    logger.info("ZK device scheduler started successfully")
                    print("ZK device scheduler started successfully")
                    return HttpResponse("<script>window.location.reload()</script>")

                except Exception as error:
                    logger.error(f"An error occurred while scheduling ZK device: {error}")
                    print(f"An error occurred while scheduling ZK device: {error}")
                    script = """
                    <script>
                        Swal.fire({
                          title : "Schedule Attendance unsuccessful",
                          text: "Please double-check the accuracy of the provided IP Address and Port Number for correctness",
                          icon: "warning",
                          showConfirmButton: false,
                          timer: 3500,
                          timerProgressBar: true,
                          didClose: () => {
                            location.reload();
                            },
                        });
                    </script>
                    """
                    return HttpResponse(script)

            elif device.machine_type == "anviz":
                try:
                    logger.debug("Processing Anviz device scheduling")
                    print("Processing Anviz device scheduling")

                    duration = request.POST.get("scheduler_duration")
                    device.is_scheduler = True
                    device.scheduler_duration = duration
                    device.save()

                    scheduler = BackgroundScheduler()
                    scheduler.add_job(
                        lambda: anviz_biometric_attendance_scheduler(device.id),
                        "interval",
                        seconds=str_time_seconds(device.scheduler_duration),
                    )
                    scheduler.start()

                    logger.info("Anviz device scheduler started successfully")
                    print("Anviz device scheduler started successfully")
                    return HttpResponse("<script>window.location.reload()</script>")
                except Exception as e:
                    logger.error(f"Anviz scheduling failed: {e}")
                    print(f"Anviz scheduling failed: {e}")
                    return HttpResponse("Error occurred in Anviz scheduler", status=500)

            elif device.machine_type == "dahua":
                try:
                    logger.debug("Processing Dahua device scheduling")
                    print("Processing Dahua device scheduling")

                    duration = request.POST.get("scheduler_duration")
                    device.is_scheduler = True
                    device.is_live = False
                    device.scheduler_duration = duration
                    device.save()

                    scheduler = BackgroundScheduler()
                    scheduler.add_job(
                        lambda: dahua_biometric_attendance_scheduler(device.id),
                        "interval",
                        seconds=str_time_seconds(device.scheduler_duration),
                    )
                    scheduler.start()

                    logger.info("Dahua device scheduler started successfully")
                    print("Dahua device scheduler started successfully")
                    return HttpResponse("<script>window.location.reload()</script>")
                except Exception as e:
                    logger.error(f"Dahua scheduling failed: {e}")
                    print(f"Dahua scheduling failed: {e}")
                    return HttpResponse("Error occurred in Dahua scheduler", status=500)

            elif device.machine_type == "cosec":
                try:
                    logger.debug("Processing Cosec device scheduling")
                    print("Processing Cosec device scheduling")

                    duration = request.POST.get("scheduler_duration")
                    device.is_scheduler = True
                    device.is_live = False
                    device.scheduler_duration = duration
                    device.save()

                    scheduler = BackgroundScheduler()
                    existing_thread = BIO_DEVICE_THREADS.get(device.id)
                    if existing_thread:
                        existing_thread.stop()
                        del BIO_DEVICE_THREADS[device.id]

                    scheduler.add_job(
                        lambda: cosec_biometric_attendance_scheduler(device.id),
                        "interval",
                        seconds=str_time_seconds(device.scheduler_duration),
                    )
                    scheduler.start()

                    logger.info("Cosec device scheduler started successfully")
                    print("Cosec device scheduler started successfully")
                    return HttpResponse("<script>window.location.reload()</script>")
                except Exception as e:
                    logger.error(f"Cosec scheduling failed: {e}")
                    print(f"Cosec scheduling failed: {e}")
                    return HttpResponse("Error occurred in Cosec scheduler", status=500)

            elif device.machine_type == "etimeoffice":
                try:
                    logger.debug("Processing ETimeOffice device scheduling")
                    print("Processing ETimeOffice device scheduling")

                    duration = request.POST.get("scheduler_duration")
                    device.is_scheduler = True
                    device.is_live = False
                    device.scheduler_duration = duration
                    device.save()

                    scheduler = BackgroundScheduler()
                    scheduler.add_job(
                        lambda: etimeoffice_biometric_attendance_scheduler(device.id),
                        "interval",
                        seconds=str_time_seconds(device.scheduler_duration),
                    )
                    scheduler.start()

                    logger.info("ETimeOffice device scheduler started successfully")
                    print("ETimeOffice device scheduler started successfully")
                    return HttpResponse("<script>window.location.reload()</script>")
                except Exception as e:
                    logger.error(f"ETimeOffice scheduling failed: {e}")
                    print(f"ETimeOffice scheduling failed: {e}")
                    return HttpResponse("Error occurred in ETimeOffice scheduler", status=500)

            else:
                logger.warning(f"Unknown machine type: {device.machine_type}")
                print(f"Unknown machine type: {device.machine_type}")
                return HttpResponse("<script>window.location.reload()</script>")

        else:
            logger.warning("Scheduler form is invalid")
            print("Scheduler form is invalid")

        context["scheduler_form"] = scheduler_form
        try:
            response = render(request, "biometric/scheduler_device_form.html", context)
            logger.debug("Rendered scheduler form for invalid submission")
            print("Rendered scheduler form for invalid submission")
            return HttpResponse(
                response.content.decode("utf-8")
                + "<script>$('#BiometricDeviceTestModal').removeClass('oh-modal--show');\
                $('#BiometricDeviceModal').toggleClass('oh-modal--show');</script>"
            )
        except Exception as e:
            logger.error(f"Error rendering invalid form: {e}")
            print(f"Error rendering invalid form: {e}")
            return HttpResponse("Error rendering form", status=500)

    logger.debug("Rendering scheduler form for GET request")
    print("Rendering scheduler form for GET request")
    try:
        return render(request, "biometric/scheduler_device_form.html", context)
    except Exception as e:
        logger.error(f"Error rendering scheduler form (GET): {e}")
        print(f"Error rendering scheduler form (GET): {e}")
        return HttpResponse("Error rendering page", status=500)


@login_required
@install_required
@hx_request_required
@permission_required("biometric.change_biometricdevices")
def biometric_device_unschedule(request, device_id):
    """
    Handles unschedule of attendance capture for a biometric device.

    Parameters:
    - request (HttpRequest): The HTTP request object.
    - device_id (uuid): The ID of the biometric device for which unscheduling is being done.

    Returns:
    - HttpResponseRedirect: Redirects to the biometric devices view after unscheduling.
    """
    previous_data = request.GET.urlencode()
    device = BiometricDevices.objects.get(id=device_id)
    device.is_scheduler = False
    device.save()
    messages.success(request, _("Biometric device unscheduled successfully"))
    return redirect(f"/biometric/search-devices?{previous_data}")


@login_required
@install_required
@hx_request_required
@permission_required("biometric.add_biometricdevices")
def biometric_device_add(request):
    """
    Handles the addition of a new biometric device.

    Parameters:
    - request (HttpRequest): The HTTP request object containing data about the request.

    Returns:
    - HttpResponse: Renders the 'add_biometric_device.html' template with the biometric device form.
    """
    previous_data = unquote(request.GET.urlencode())[len("pd=") :]
    biometric_form = BiometricDeviceForm()
    if request.method == "POST":
        biometric_form = BiometricDeviceForm(request.POST)
        if biometric_form.is_valid():
            biometric_form.save()
            messages.success(request, _("Biometric device added successfully."))
            biometric_form = BiometricDeviceForm()
    context = {"biometric_form": biometric_form, "pd": previous_data}
    return render(request, "biometric/add_biometric_device.html", context)


@login_required
@install_required
@hx_request_required
@permission_required("biometric.change_biometricdevices")
def biometric_device_edit(request, device_id):
    """
    Handles the editing of an existing biometric device.

    Parameters:
    - request (HttpRequest): The HTTP request object containing data about the request.
    - device_id (uuid): The ID of the biometric device to be edited.

    Returns:
    - HttpResponse: Renders the 'edit_biometric_device.html' template with the biometric
                    device form pre-filled with existing data.
    """
    device = BiometricDevices.find(device_id)
    if not device:
        messages.error(request, _("Biometric device not found."))
        return render(request, "biometric/edit_biometric_device.html")
    biometric_form = BiometricDeviceForm(instance=device)
    if request.method == "POST":
        biometric_form = BiometricDeviceForm(request.POST, instance=device)
        if biometric_form.is_valid():
            biometric_form.save()
            messages.success(request, _("Biometric device updated successfully."))
    context = {
        "biometric_form": biometric_form,
        "device_id": device_id,
    }
    return render(request, "biometric/edit_biometric_device.html", context)


@login_required
@install_required
@hx_request_required
@permission_required("biometric.change_biometricdevices")
def biometric_device_archive(request, device_id):
    """
    This method is used to archive or un-archive devices
    """
    previous_data = request.GET.urlencode()
    device_obj = BiometricDevices.find(device_id)
    if not device_obj:
        messages.error(request, _("Biometric device not found."))
        return redirect(f"/biometric/search-devices?{previous_data}")
    device_obj.is_active = not device_obj.is_active
    device_obj.save()
    message = _("archived") if not device_obj.is_active else _("un-archived")
    messages.success(request, _("Device is %(message)s") % {"message": message})
    return redirect(f"/biometric/search-devices?{previous_data}")


@login_required
@install_required
@hx_request_required
@permission_required("biometric.delete_biometricdevices")
def biometric_device_delete(request, device_id):
    """
    Handles the deletion of a biometric device.

    Parameters:
    - request (HttpRequest): The HTTP request object containing data about the request.
    - device_id (uuid): The ID of the biometric device to be deleted.

    Returns:
    - HttpResponseRedirect: Redirects to the 'search-devices' page after deleting the
                            biometric device.

    """
    previous_data = request.GET.urlencode()
    device_obj = BiometricDevices.find(device_id)
    if not device_obj:
        messages.error(request, _("Biometric device not found."))
        return redirect(f"/biometric/search-devices?{previous_data}")
    device_obj.delete()
    messages.success(request, _("Biometric device deleted successfully."))
    return redirect(f"/biometric/search-devices?{previous_data}")


@login_required
@install_required
@hx_request_required
@permission_required("biometric.view_biometricdevices")
def search_devices(request):
    """
    This method is used to search biometric device model and return matching objects
    """
    previous_data = request.GET.urlencode()
    search = request.GET.get("search")
    is_active = request.GET.get("is_active")
    if search is None:
        search = ""
    devices = BiometricDeviceFilter(request.GET).qs.order_by("-created_at")
    if not is_active or is_active == "unknown":
        devices = devices.filter(is_active=True)
    data_dict = []
    data_dict = parse_qs(previous_data)
    get_key_instances(BiometricDevices, data_dict)
    template = "biometric/card_biometric_devices.html"
    if request.GET.get("view") == "list":
        template = "biometric/list_biometric_devices.html"

    devices = paginator_qry(devices, request.GET.get("page"))
    return render(
        request,
        template,
        {
            "devices": devices,
            "pd": previous_data,
            "filter_dict": data_dict,
        },
    )


def render_connection_response(title, text, icon):
    """
    Helper function to render the connection
    response from device test connection.
    """
    context = {
        "title": title,
        "text": text,
        "icon": icon,
    }
    return render_to_string("biometric/test_connection_script.html", context)


def test_zkteco_connection(device):
    """Test connection for ZKTeco device."""
    conn = None
    port_no = device.port
    machine_ip = device.machine_ip
    password = device.zk_password
    zk_device = ZK(
        machine_ip,
        port=port_no,
        timeout=60,
        password=int(password),
        force_udp=False,
        ommit_ping=False,
    )
    try:
        conn = zk_device.connect()
        conn.test_voice(index=0)
        find_employees_in_zk(device.id)
        return render_connection_response(
            _("Connection Successful"),
            _("ZKTeco test connection successful."),
            "success",
        )
    except zk_exception.ZKErrorResponse:
        return render_connection_response(
            _("Authentication Error"),
            _("Double-check the provided IP, Port, and Password."),
            "warning",
        )
    except Exception:
        logger.error("ZKTeco connection error", exc_info=True)
        return render_connection_response(
            _("Connection unsuccessful"),
            _("Please check the IP, Port, and Password."),
            "warning",
        )
    finally:
        if conn is not None:
            conn.disconnect()


def test_anviz_connection(device):
    """Test connection for Anviz device."""

    try:
        from .anviz import CrossChexCloudAPI

        anviz = CrossChexCloudAPI(
            api_url=device.api_url,
            api_key=device.api_key,
            api_secret=device.api_secret,
            anviz_request_id=device.anviz_request_id,
        )
        test_response = anviz.test_connection()

        if test_response.get("token"):
            device.api_token = test_response.get("token")
            device.api_expires = test_response.get("expires")
            device.save()
            records = anviz.get_attendance_records()

            return render_connection_response(
                _("Connection Successful"),
                _("Anviz test connection successful."),
                "success",
            )
        else:
            return render_connection_response(
                _("Connection unsuccessful"),
                _("API credentials might be incorrect."),
                "warning",
            )
    except Exception as error:
        logger.error("Anviz connection error", exc_info=True)
        return render_connection_response(
            _("Connection unsuccessful"), _("API request failed."), "warning"
        )


def test_cosec_connection(device):
    """Test connection for COSEC device."""
    cosec = COSECBiometric(
        device.machine_ip,
        device.port,
        device.bio_username,
        device.bio_password,
        timeout=10,
    )
    response = cosec.basic_config()
    if response.get("app"):
        find_employees_in_cosec(device.id)
        return render_connection_response(
            _("Connection Successful"),
            _("Matrix test connection successful."),
            "success",
        )
    else:
        return render_connection_response(
            _("Connection unsuccessful"),
            _("Double-check the provided Machine IP, Username, and Password."),
            "warning",
        )


def test_dahua_connection(device):
    """Test connection for Dahua device."""
    dahua = DahuaAPI(
        ip=device.machine_ip,
        username=device.bio_username,
        password=device.bio_password,
    )
    response = dahua.get_system_info()
    if response.get("status_code") == 200:
        return render_connection_response(
            _("Connection Successful"),
            _("Dahua test connection successful."),
            "success",
        )
    else:
        return render_connection_response(
            _("Connection unsuccessful"),
            _("Double-check the provided Machine IP, Username, and Password."),
            "warning",
        )


def test_etimeoffice_connection(device):
    """Test connection for e-TimeOffice device."""
    now = datetime.now()
    etimeoffice = ETimeOfficeAPI(
        username=device.bio_username,
        password=device.bio_password,
    )

    from_date = f"{now.day:02d}/{now.month:02d}/{now.year}_00:00"
    to_date = (
        f"{now.day:02d}/{now.month:02d}/{now.year}_{now.hour:02d}:{now.minute:02d}"
    )

    try:
        response = etimeoffice.download_punch_data(from_date=from_date, to_date=to_date)
        if response.get("Msg") == "Success":
            return render_connection_response(
                _("Connection Successful"),
                _("e-Time Office test connection successful."),
                "success",
            )

        error_msg = response.get("Message")
        return render_connection_response(
            _("Connection unsuccessful"),
            _("Double-check the provided API Url, Username, and Password: {}").format(
                error_msg
            ),
            "warning",
        )

    except Exception as e:
        return render_connection_response(
            _("Connection error"),
            _(f"API request failed with exception: {str(e)}"),
            "danger",
        )


@login_required
@install_required
@hx_request_required
@permission_required("biometric.view_biometricdevices")
def biometric_device_test(request, device_id):
    """
    Test the connection with the specified biometric device.
    """

    # Retrieve device and validate
    device = BiometricDevices.objects.filter(id=device_id).first()
    if not device:
        return HttpResponse("Device not found.", status=404)

    script = ""
    try:
        if device.machine_type == "zk":
            script = test_zkteco_connection(device)
        elif device.machine_type == "anviz":
            script = test_anviz_connection(device)
        elif device.machine_type == "cosec":
            script = test_cosec_connection(device)
        elif device.machine_type == "dahua":
            script = test_dahua_connection(device)
        elif device.machine_type == "etimeoffice":
            script = test_etimeoffice_connection(device)
        else:
            script = render_connection_response(
                "Connection unsuccessful",
                "Please select a valid biometric device.",
                "warning",
            )
    except Exception as error:
        logger.error("Error in biometric_device_test", exc_info=True)
        script = render_connection_response(
            "Connection unsuccessful", "An unexpected error occurred.", "warning"
        )

    return HttpResponse(script)


@login_required
@install_required
@hx_request_required
@permission_required("biometric.view_biometricdevices")
def biometric_device_fetch_logs(request, device_id):
    """
    Fetches biometric attendance logs from a specified device.

    This view function connects to a biometric device based on the device type (zk, anviz, cosec, dahua, or etimeoffice)
    and retrieves the attendance logs.
    """
    device = BiometricDevices.find(device_id)
    if not device:
        return HttpResponse("Device not found.", status=404)
    script = ""
    if device.machine_type == "zk":
        attendance_count, error_message = zk_biometric_attendance_logs(device)
        if isinstance(attendance_count, int):
            script = render_connection_response(
                _("Logs Fetched Successfully"),
                _(
                    f"Biometric attendance logs fetched successfully. Total records: {attendance_count}"
                ),
                "success",
            )
        elif "Authentication" in error_message:
            script = render_connection_response(
                _("Authentication Error"),
                _("Double-check the provided IP, Port, and Password."),
                "warning",
            )
        else:
            script = render_connection_response(
                _("Connection Unsuccessful"),
                _(f"Please check the IP, Port, and Password. Error: {error_message}"),
                "warning",
            )
    elif device.machine_type == "anviz":
        attendance_count = anviz_biometric_attendance_logs(device)
        if isinstance(attendance_count, int):
            script = render_connection_response(
                _("Logs Fetched Successfully"),
                _(
                    f"Biometric attendance logs fetched successfully. Total records: {attendance_count}"
                ),
                "success",
            )
        else:
            script = render_connection_response(
                _("Connection unsuccessful"),
                _("API credentials might be incorrect."),
                "warning",
            )

    elif device.machine_type == "cosec":
        attendance_count = cosec_biometric_attendance_logs(device)
        if isinstance(attendance_count, int):
            script = render_connection_response(
                _("Logs Fetched Successfully"),
                _(
                    f"Biometric attendance logs fetched successfully. Total records: {attendance_count}"
                ),
                "success",
            )
        else:
            script = render_connection_response(
                _("Connection unsuccessful"),
                _("Double-check the provided Machine IP, Username, and Password."),
                "warning",
            )
    elif device.machine_type == "dahua":
        attendance_count = dahua_biometric_attendance_logs(device)
        if isinstance(attendance_count, int):
            script = render_connection_response(
                _("Logs Fetched Successfully"),
                _(
                    f"Biometric attendance logs fetched successfully. Total records: {attendance_count}"
                ),
                "success",
            )
        else:
            script = render_connection_response(
                _("Connection unsuccessful"),
                _("Double-check the provided Machine IP, Username, and Password."),
                "warning",
            )
    elif device.machine_type == "etimeoffice":
        attendance_count = etimeoffice_biometric_attendance_logs(device)
        if isinstance(attendance_count, int):
            script = render_connection_response(
                _("Logs Fetched Successfully"),
                _(
                    f"Biometric attendance logs fetched successfully. Total records: {attendance_count}"
                ),
                "success",
            )
        else:
            script = render_connection_response(
                _("Connection unsuccessful"),
                _("Double-check the provided API Url, Username, and Password"),
                "warning",
            )
    else:
        script = render_connection_response(
            "Connection unsuccessful",
            "Please select a valid biometric device.",
            "warning",
        )

    return HttpResponse(script)


def zk_employees_fetch(device):
    """
    Fetch employee data from the specified ZK biometric device.
    Now includes ZKBiotime Web API fallback.

    Parameters:
    - device: Biometric device object containing machine IP, port, etc.

    Returns:
    - list: A list of employee objects with associated data
    """
    logger.info(f"Starting zk_employees_fetch for device {device.id}")
    employees = []
    
    # First try direct ZK connection
    try:
        logger.debug(f"Attempting direct ZK connection to {device.machine_ip}:{device.port}")
        zk_device = ZK(
            device.machine_ip,
            port=device.port,
            timeout=5,
            password=int(device.zk_password),
            force_udp=False,
            ommit_ping=False,
        )
        
        try:
            conn = zk_device.connect()
            if not conn:
                logger.warning("Failed to establish direct ZK connection")
                raise Exception("Connection failed")
                
            logger.info("Successfully connected to ZK device")
            
            try:
                conn.enable_device()
                users = conn.get_users()
                fingers = conn.get_templates()
                logger.debug(f"Retrieved {len(users)} users and {len(fingers)} fingerprint templates")
                
                # Process users with direct connection data
                bio_employees = BiometricEmployees.objects.filter(device_id=device)
                bio_lookup = {bio.user_id: bio for bio in bio_employees}
                
                for user in users:
                    user_id = user.user_id
                    uid = user.uid
                    bio_id = bio_lookup.get(user_id)
                    
                    if bio_id:
                        employee = bio_id.employee_id
                        employee_work_info = EmployeeWorkInformation.objects.filter(
                            employee_id=employee
                        ).first()
                        
                        # Add employee info
                        user.__dict__.update({
                            "work_email": getattr(employee_work_info, 'email', None),
                            "phone": getattr(employee_work_info, 'mobile', None),
                            "job_position": getattr(employee_work_info, 'job_position_id', None),
                            "employee": employee,
                            "badge_id": employee.badge_id,
                        })
                        
                        # Add fingerprint info
                        finger_print = [
                            finger.fid for finger in fingers 
                            if finger.uid == uid
                        ]
                        user.__dict__["finger"] = finger_print
                        
                        employees.append(user)
                
                logger.info(f"Successfully processed {len(employees)} employees from direct connection")
                return employees
                
            except Exception as process_error:
                logger.error(f"Error processing device data: {process_error}")
                raise
            finally:
                try:
                    conn.disconnect()
                    logger.debug("Disconnected from ZK device")
                except Exception as disconnect_error:
                    logger.warning(f"Error disconnecting: {disconnect_error}")
                
        except Exception as conn_error:
            logger.error(f"Direct connection error: {conn_error}")
            raise
            
    except Exception as direct_error:
        logger.warning(f"Direct ZK access failed: {direct_error}, trying API fallback")
        
        # Try ZKBiotime API fallback
        try:
            # Hardcoded API configuration (same as in ZKBioAttendance)
            hardcoded_api_url = ""
            hardcoded_username = ""
            hardcoded_password = ""
            
            # Get API token
            token = get_zkbiotime_token(
                hardcoded_api_url,
                hardcoded_username,
                hardcoded_password
            )
            
            if not token:
                logger.error("Failed to get API token")
                return []
                
            # Fetch employees from API
            headers = {
                "Authorization": f"Token {token}",
                "Content-Type": "application/json"
            }
            
            try:
                employees_url = f"{hardcoded_api_url}/personnel/api/employees/"
                response = requests.get(
                    employees_url, 
                    headers=headers,
                    params={"page_size": 100},
                    timeout=10
                )
                
                if response.status_code == 200:
                    api_data = response.json()
                    api_employees = api_data.get("data", [])
                    logger.info(f"Retrieved {len(api_employees)} employees from API")
                    
                    # Create mock User objects to match ZK format
                    class ApiUser:
                        def __init__(self, **kwargs):
                            for key, value in kwargs.items():
                                setattr(self, key, value)
                    
                    # Get all biometric employees for this device
                    bio_employees = BiometricEmployees.objects.filter(device_id=device)
                    bio_lookup = {bio.user_id: bio for bio in bio_employees}
                    
                    for emp_data in api_employees:
                        emp_code = emp_data.get("emp_code")
                        
                        # Create API user object with same attributes as ZK user
                        api_user = ApiUser(
                            uid=emp_data.get("id"),  # Use API id as uid
                            user_id=emp_code,
                            name=f"{emp_data.get('first_name', '')} {emp_data.get('last_name', '')}".strip()
                        )
                        
                        # Add employee info if mapped in BiometricEmployees
                        bio_id = bio_lookup.get(emp_code)
                        logger.debug(f"Processing employee {emp_code} with bio_id: {bio_id}")
                        if bio_id:
                            employee = bio_id.employee_id
                            employee_work_info = EmployeeWorkInformation.objects.filter(
                                employee_id=employee
                            ).first()
                            
                            # Add additional attributes
                            api_user.employee = employee
                            api_user.badge_id = employee.badge_id if employee else None
                            api_user.work_email = getattr(employee_work_info, 'email', None)
                            api_user.phone = getattr(employee_work_info, 'mobile', None)
                            api_user.job_position = getattr(employee_work_info, 'job_position_id', None)
                            
                            # Add fingerprint info from API response
                            api_user.finger = []
                            if emp_data.get('fingerprint'):
                                api_user.finger = [1]  # Indicate fingerprint exists
                                
                            employees.append(api_user)
                    
                    logger.info(f"Successfully processed {len(employees)} employees from API")
                    return employees
                else:
                    logger.error(f"API request failed: {response.status_code}")
                    return []
                    
            except requests.exceptions.RequestException as api_error:
                logger.error(f"API request error: {api_error}")
                return []
                
        except Exception as api_error:
            logger.error(f"API fallback error: {api_error}")
            return []
    
    return employees  # Return empty list if all methods fail

def cosec_employee_fetch(device_id):
    """
    Fetch employee data from the COSEC biometric device associated with the specified device ID.

    Parameters:
    - device_id: ID of the biometric device.

    Returns:
    - list: A list of dictionaries, where each dictionary represents an employee with associated
            data including user ID, employee ID, finger count, and card count.
    """
    users = []
    device = BiometricDevices.objects.get(id=device_id)
    employees = BiometricEmployees.objects.filter(device_id=device)
    cosec = COSECBiometric(
        device.machine_ip, device.port, device.bio_username, device.bio_password
    )
    for employee in employees:
        user = cosec.get_cosec_user(user_id=employee.user_id)
        if user.get("user-id"):
            user["employee_id"] = employee.employee_id
            user_credential = cosec.get_user_credential_count(user_id=employee.user_id)
            user["finger-count"] = user_credential.get("finger-count")
            user["face-count"] = user_credential.get("face-count")
            user["card-count"] = user_credential.get("card-count")
            new_dict = {}
            for key, value in user.items():
                new_key = key.replace("-", "_")
                new_dict[new_key] = value
            users.append(new_dict)
        else:
            BiometricEmployees.objects.get(id=employee.id).delete()
    return users


def find_employees_in_cosec(device_id):
    """
    Synchronize active employees with a COSEC biometric device.

    This function retrieves a list of active employees from the database,
    checks their presence on a specified COSEC biometric device, and updates
    the database with employees who are registered on the COSEC device.

    Args:
        device_id (uuid): The ID of the biometric device to synchronize with.
    """
    device = BiometricDevices.objects.get(id=device_id)
    employees = Employee.objects.filter(is_active=True).values_list("id", "badge_id")
    cosec = COSECBiometric(
        device.machine_ip, device.port, device.bio_username, device.bio_password
    )
    existing_user_ids = BiometricEmployees.objects.filter(device_id=device).values_list(
        "user_id", flat=True
    )
    biometric_employees_to_create = []
    for employee_id, badge_id in employees:
        if badge_id and badge_id.isalnum() and len(badge_id) <= 15:
            user = cosec.get_cosec_user(user_id=badge_id)
            if user.get("user-id") and user.get("user-id") not in existing_user_ids:
                biometric_employees_to_create.append(
                    BiometricEmployees(
                        ref_user_id=user.get("ref-user-id"),
                        user_id=user.get("user-id"),
                        employee_id_id=employee_id,
                        device_id_id=device_id,
                    )
                )
    BiometricEmployees.objects.bulk_create(biometric_employees_to_create)


def find_employees_in_zk(device_id):
    """
    Synchronize active employees with a COSEC biometric device.

    This function retrieves a list of active employees from the database,
    checks their presence on a specified COSEC biometric device, and updates
    the database with employees who are registered on the COSEC device.

    Args:
        device_id (uuid): The ID of the biometric device to synchronize with.
    """
    device = BiometricDevices.objects.get(id=device_id)
    employees = Employee.objects.filter(is_active=True).values_list("id", "badge_id")
    existing_user_ids = set(
        BiometricEmployees.objects.filter(device_id=device_id).values_list(
            "user_id", flat=True
        )
    )
    zk_device = ZK(
        device.machine_ip, port=device.port, password=int(device.zk_password), timeout=5
    )
    conn = zk_device.connect()
    zk_users = {user.user_id: user.uid for user in conn.get_users()}
    biometric_employees_to_create = [
        BiometricEmployees(
            uid=zk_users[badge_id],
            user_id=badge_id,
            employee_id_id=employee_id,
            device_id_id=device_id,
        )
        for employee_id, badge_id in employees
        if badge_id and badge_id in zk_users and badge_id not in existing_user_ids
    ]
    BiometricEmployees.objects.bulk_create(biometric_employees_to_create)
    conn.disconnect()


@login_required
@install_required
@permission_required("biometric.view_biometricemployees")
def biometric_device_employees(request, device_id, **kwargs):
    """
    View function to display employees associated with a biometric device.

    Depending on the machine type of the biometric device (either "zk" or "cosec"),
    this function fetches the relevant employees and renders the appropriate template.

    Args:
        request (HttpRequest): The HTTP request object.
        device_id (uuid): The ID of the biometric device.
        **kwargs: Additional keyword arguments.

    Returns:
        HttpResponse: The rendered template response or a redirect to `biometric_devices_view`
                      in case of an error.
    """
    previous_data = request.GET.urlencode()
    device = BiometricDevices.find(device_id)
    if device:
        try:
            if device.machine_type == "zk":
                employee_add_form = EmployeeBiometricAddForm()
                employees = zk_employees_fetch(device)
                employees = paginator_qry(employees, request.GET.get("page"))
                context = {
                    "employees": employees,
                    "device_id": device_id,
                    "device": device,
                    "form": employee_add_form,
                    "pd": previous_data,
                }
                return render(
                    request, "biometric/view_employees_biometric.html", context
                )
            if device.machine_type == "cosec":
                employee_add_form = CosecUserAddForm()
                employees = cosec_employee_fetch(device_id)
                employees = biometric_paginator_qry(
                    employees, int(request.GET.get("page", 1))
                )
                context = {
                    "employees": employees,
                    "device_id": device.id,
                    "form": employee_add_form,
                    "pd": previous_data,
                }
                return render(request, "biometric/view_cosec_employees.html", context)
            if device.machine_type == "dahua":
                employees = BiometricEmployees.objects.filter(device_id=device_id)
                context = {
                    "device_id": device.id,
                    "employees": employees,
                }
                return render(
                    request, "biometric_users/dahua/view_dahua_employees.html", context
                )
            if device.machine_type == "etimeoffice":
                employees = BiometricEmployees.objects.filter(device_id=device_id)
                context = {
                    "device_id": device.id,
                    "employees": employees,
                }
                return render(
                    request,
                    "biometric_users/etimeoffice/view_etimeoffice_employees.html",
                    context,
                )
        except Exception as error:
            logger.error(f"An error occurred: {error}")
            messages.info(
                request,
                _(
                    "Failed to establish a connection. Please verify the accuracy of the IP\
                    Address , Port No. and Password of the device."
                ),
            )
    else:
        messages.error(request, _("Biometric device not found"))
    return redirect(biometric_devices_view)


@login_required
@install_required
@hx_request_required
@permission_required("biometric.view_biometricemployees")
def search_employee_device(request):
    """
    View function to search for employees associated with a specific biometric device.

    This function handles searching employees based on their first name and the type of
    biometric device (either "zk" or "cosec"). It then renders the appropriate template
    with the filtered employee list.

    Args:
        request (HttpRequest): The HTTP request object.

    Returns:
        HttpResponse: The rendered template response with the context.
    """
    previous_data = request.GET.urlencode()
    device_id = request.GET.get("device")
    device = BiometricDevices.objects.get(id=device_id)
    search = request.GET.get("search")
    if device.machine_type == "zk":
        employees = zk_employees_fetch(device)
        if search:
            search_employees = BiometricEmployees.objects.filter(
                employee_id__employee_first_name__icontains=search
            )
            search_uids = search_employees.values_list("uid", flat=True)
            employees = [
                employee for employee in employees if employee.uid in search_uids
            ]
        employees = paginator_qry(employees, request.GET.get("page"))
        template = "biometric/list_employees_biometric.html"
        context = {
            "employees": employees,
            "device_id": device_id,
            "pd": previous_data,
        }
    elif device.machine_type == "dahua" or device.machine_type == "etimeoffice":
        search_employees = BiometricEmployees.objects.filter(device_id=device)
        if search:
            search_employees = BiometricEmployees.objects.filter(
                employee_id__employee_first_name__icontains=search, device_id=device
            )
        template = (
            "biometric_users/dahua/list_dahua_employees.html"
            if device.machine_type == "dahua"
            else "biometric_users/etimeoffice/list_etimeoffice_employees.html"
        )
        context = {
            "device_id": device.id,
            "employees": search_employees,
        }

    else:
        employees = cosec_employee_fetch(device_id)
        if search:
            search_employees = BiometricEmployees.objects.filter(
                employee_id__employee_first_name__icontains=search, device_id=device
            )
        else:
            search_employees = BiometricEmployees.objects.filter(device_id=device)
        queryset_user_ids = [employee.user_id for employee in search_employees]
        filtered_employees = [
            employee
            for employee in employees
            if employee["user_id"] in queryset_user_ids
        ]
        filtered_employees = biometric_paginator_qry(
            filtered_employees, int(request.GET.get("page", 1))
        )
        template = "biometric/list_employees_cosec_biometric.html"
        context = {
            "employees": filtered_employees,
            "device_id": device_id,
            "pd": previous_data,
        }
    return render(request, template, context)


@login_required
@install_required
@permission_required("biometric.delete_biometricemployees")
def delete_biometric_user(request, uid, device_id):
    """
    This function connects to the specified biometric device, deletes the user
    identified by the given UID, and removes the corresponding entry from the
    BiometricEmployees table in the local database

    Args:
        request (HttpRequest): The HTTP request object.
        uid (str): The UID of the user to be deleted from the biometric device.
        device_id (uuid): The ID of the biometric device.

    Returns:
        HttpResponse: A redirect response to the list of employees for the specified
                      biometric device.
    """
    device = BiometricDevices.objects.get(id=device_id)
    zk_device = ZK(
        device.machine_ip,
        port=device.port,
        timeout=5,
        password=int(device.zk_password),
        force_udp=False,
        ommit_ping=False,
    )
    conn = zk_device.connect()
    conn.delete_user(uid=uid)
    employee_bio = BiometricEmployees.objects.filter(uid=uid).first()
    employee_bio.delete()
    messages.success(
        request,
        _("{} successfully removed from the biometric device.").format(
            employee_bio.employee_id
        ),
    )
    redirect_url = f"/biometric/biometric-device-employees/{device_id}/"
    return redirect(redirect_url)


@login_required
@install_required
@permission_required("biometric.change_biometricemployees")
def enable_cosec_face_recognition(request, user_id, device_id):
    """
    View function to enable face recognition for a user on a COSEC biometric device

    Args:
        request (HttpRequest): The HTTP request object.
        user_id (str): The ID of the user for whom face recognition is to be enabled.
        device_id (uuid): The ID of the COSEC biometric device.

    Returns:
        HttpResponse: A redirect response to the list of employees for the specified
                      biometric device.
    """
    device = BiometricDevices.find(device_id)
    if device:
        cosec = COSECBiometric(
            device.machine_ip,
            device.port,
            device.bio_username,
            device.bio_password,
        )
        enable_fr = cosec.enable_user_face_recognition(user_id=user_id, enable_fr=True)
        response_code = enable_fr.get("Response-Code")
        if response_code == "0":
            messages.success(request, _("Face recognition enabled successfully"))
        else:
            messages.error(request, _("Something went wrong when enabling face"))
    else:
        messages.error(request, _("Device not found"))
    return redirect(f"/biometric/biometric-device-employees/{device_id}/")


@login_required
@install_required
@hx_request_required
@permission_required("biometric.change_biometricemployees")
def edit_cosec_user(request, user_id, device_id):
    """
    View function to edit the details of a COSEC biometric user.

    Args:
        request (HttpRequest): The HTTP request object.
        user_id (str): The ID of the user to be edited.
        device_id (uuid): The ID of the COSEC biometric device.

    Returns:
        HttpResponse: The rendered form template for GET requests, and a response with
                      a success message for valid POST requests. Reloads the page after
                      successful update.

    """
    device = BiometricDevices.objects.get(id=device_id)
    cosec = COSECBiometric(
        device.machine_ip,
        device.port,
        device.bio_username,
        device.bio_password,
    )
    user = cosec.get_cosec_user(user_id)
    if user.get("name"):
        year = int(user["validity-date-yyyy"])
        month = int(user["validity-date-mm"])
        day = int(user["validity-date-dd"])
        date_object = datetime(year, month, day)
        formatted_date = date_object.strftime("%Y-%m-%d")
        initial_data = {
            "name": user["name"],
            "user_active": bool(int(user["user-active"])),
            "vip": bool(int(user["vip"])),
            "validity_enable": bool(int(user["validity-enable"])),
            "validity_end_date": formatted_date,
        }

        if "by-pass-finger" in user:
            initial_data["by_pass_finger"] = bool(int(user["by-pass-finger"]))

        form = COSECUserForm(initial=initial_data)

        if request.method == "POST":
            form = COSECUserForm(request.POST)
            if form.is_valid():
                name = form.cleaned_data["name"]
                user_active = form.cleaned_data["user_active"]
                vip = form.cleaned_data["vip"]
                validity_enable = form.cleaned_data["validity_enable"]
                validity_end_date_str = str(form.cleaned_data["validity_end_date"])
                validity_end_date = datetime.strptime(
                    validity_end_date_str, "%Y-%m-%d"
                ).date()
                validity_year = validity_end_date.year
                validity_month = validity_end_date.month
                validity_day = validity_end_date.day
                by_pass_finger = form.cleaned_data["by_pass_finger"]
                update_user = cosec.set_cosec_user(
                    user_id=user["user-id"],
                    ref_user_id=user["ref-user-id"],
                    name=name,
                    user_active=int(user_active),
                    vip=int(vip),
                    by_pass_finger=int(by_pass_finger),
                    validity_enable=int(validity_enable),
                    validity_date_dd=validity_day,
                    validity_date_mm=validity_month,
                    validity_date_yyyy=validity_year,
                )
                if (
                    update_user.get("Response-Code")
                    and update_user.get("Response-Code") == "0"
                ):
                    messages.success(
                        request, _("Biometric user data updated successfully")
                    )
                    return HttpResponse("<script>window.location.reload()</script>")
                if update_user.get("error"):
                    error = update_user.get("error")
                    if "validity-date-yyyy" in error:
                        form.add_error(
                            None,
                            _(
                                "This date cannot be used as the Validity End Date for\
                                the COSEC Biometric."
                            ),
                        )
        return render(
            request,
            "biometric/edit_cosec_user.html",
            context={"form": form, "user_id": user_id, "device_id": device_id},
        )


@login_required
@install_required
@permission_required("biometric.delete_biometricemployees")
def delete_horilla_cosec_user(request, user_id, device_id):
    """
    View function to delete a user from a COSEC biometric device and database.

    Args:
        request (HttpRequest): The HTTP request object.
        user_id (str): The ID of the user to be deleted from the COSEC biometric device.
        device_id (uuid): The ID of the COSEC biometric device.

    Returns:
        HttpResponse: A redirect response to the list of employees for the specified
                      biometric device.
    """
    device = BiometricDevices.find(device_id)
    if device:
        employee_bio = BiometricEmployees.objects.filter(
            user_id=user_id, device_id=device
        ).first()
        cosec = COSECBiometric(
            device.machine_ip,
            device.port,
            device.bio_username,
            device.bio_password,
        )
        response = cosec.delete_cosec_user(user_id)
        if response.get("Response-Code") and response.get("Response-Code") == "0":
            employee_bio.delete()
            messages.success(
                request,
                _("{} successfully removed from the biometric device.").format(
                    employee_bio.employee_id
                ),
            )
        else:
            messages.error(request, _("Biometric user not found"))
    else:
        messages.error(request, _("Biometric device not found"))
    redirect_url = (
        f"/biometric/biometric-device-employees/{device_id}/"
        if device
        else "/biometric/view-biometric-devices/"
    )
    return redirect(redirect_url)


@login_required
@install_required
@permission_required("biometric.delete_biometricemployees")
def bio_users_bulk_delete(request):
    """
    View function to delete multiple users from a ZK biometric device and the local database.

    Args:
        request (HttpRequest): The HTTP request object.

    Returns:
        JsonResponse: A JSON response indicating the success of the bulk delete operation.

    """
    conn = None
    json_ids = request.POST["ids"]
    device_id = request.POST["deviceId"]
    ids = json.loads(json_ids)
    device = BiometricDevices.objects.get(id=device_id)
    try:
        zk_device = ZK(
            device.machine_ip,
            port=device.port,
            timeout=5,
            password=int(device.zk_password),
            force_udp=False,
            ommit_ping=False,
        )
        conn = zk_device.connect()
        for user_id in ids:
            user_id = int(user_id)
            conn.delete_user(user_id=user_id)
            employee_bio = BiometricEmployees.objects.filter(user_id=user_id).first()
            employee_bio.delete()
            conn.refresh_data()
            messages.success(
                request,
                _("{} successfully removed from the biometric device.").format(
                    employee_bio.employee_id
                ),
            )
    except Exception as error:
        logger.error(f"An error occurred: {error}")
    return JsonResponse({"messages": "Success"})


@login_required
@install_required
@permission_required("biometric.delete_biometricemployees")
def cosec_users_bulk_delete(request):
    """
    View function to delete multiple users from a COSEC biometric device and database.

    Args:
        request (HttpRequest): The HTTP request object.

    Returns:
        JsonResponse: A JSON response indicating the success of the bulk delete operation.
    """
    json_ids = request.POST["ids"]
    device_id = request.POST["deviceId"]
    ids = json.loads(json_ids)
    device = BiometricDevices.objects.get(id=device_id)
    try:
        cosec = COSECBiometric(
            device.machine_ip,
            device.port,
            device.bio_username,
            device.bio_password,
        )
        for user_id in ids:
            cosec.delete_cosec_user(user_id=user_id)
            employee_bio = BiometricEmployees.objects.filter(
                user_id=user_id, device_id=device
            ).first()
            if employee_bio:
                employee_bio.delete()
            messages.success(
                request,
                f"{employee_bio.employee_id} "
                + _("successfully removed from the biometric device."),
            )

    except Exception as error:
        logger.error(f"An error occurred: {error}")
    return JsonResponse({"messages": "Success"})


@login_required
@install_required
@hx_request_required
@permission_required("biometric.add_biometricemployees")
def add_biometric_user(request, device_id):
    """
    View function to add a new user to a biometric device.

    This function adds a new user to the specified biometric device and stores their
    information in the database.

    Args:
        request (HttpRequest): The HTTP request object.
        device_id (uuid): The ID of the biometric device.

    Returns:
        HttpResponse: A JavaScript script to reload the current page after adding the user.

    """
    device = BiometricDevices.objects.get(id=device_id)
    employee_add_form = (
        EmployeeBiometricAddForm()
        if device.machine_type == "zk"
        else CosecUserAddForm()
    )
    if request.method == "POST":
        device = BiometricDevices.objects.get(id=device_id)
        try:
            if device.machine_type == "zk":
                zk_device = ZK(
                    device.machine_ip,
                    port=device.port,
                    timeout=5,
                    password=int(device.zk_password),
                    force_udp=False,
                    ommit_ping=False,
                )
                conn = zk_device.connect()
                conn.enable_device()
                existing_uids = [user.uid for user in conn.get_users()]
                existing_user_ids = [user.user_id for user in conn.get_users()]
                uid = 1
                user_id = 1000
                employee_ids = request.POST.getlist("employee_ids")
                for obj_id in employee_ids:
                    employee = Employee.objects.get(id=obj_id)
                    existing_biometric_employee = BiometricEmployees.objects.filter(
                        employee_id=employee, device_id=device
                    ).first()
                    if existing_biometric_employee is None:
                        while uid in existing_uids or user_id in existing_user_ids:
                            user_id = int(user_id)
                            uid += 1
                            user_id += 1
                        existing_uids.append(uid)
                        existing_user_ids.append(user_id)
                        employee_name = employee.get_full_name()
                        conn.set_user(
                            uid=uid,
                            name=employee_name,
                            password="",
                            group_id="",
                            user_id=str(user_id),
                            card=0,
                        )
                        # The ZK Biometric user ID must be a character value
                        # that can be converted to an integer.
                        BiometricEmployees.objects.create(
                            uid=uid,
                            user_id=str(user_id),
                            employee_id=employee,
                            device_id=device,
                        )
                        messages.success(
                            request,
                            _("{} added to biometric device successfully").format(
                                employee
                            ),
                        )
                    else:
                        messages.info(
                            request,
                            _("{} already added to biometric device").format(employee),
                        )
            else:
                cosec = COSECBiometric(
                    device.machine_ip,
                    device.port,
                    device.bio_username,
                    device.bio_password,
                )
                basic = cosec.basic_config()
                if basic.get("app"):
                    employee_ids = request.POST.getlist("employee_ids")
                    cosec_users = BiometricEmployees.objects.filter(device_id=device_id)
                    existing_ref_user_ids = list(
                        cosec_users.values_list("ref_user_id", flat=True)
                    )
                    for obj_id in employee_ids:
                        employee = Employee.objects.get(id=obj_id)
                        employee_name = employee.get_full_name()
                        user_id = employee.badge_id
                        ref_user_id = 100
                        while ref_user_id in existing_ref_user_ids:
                            ref_user_id += 1
                        existing_ref_user_ids.append(ref_user_id)
                        user = cosec.set_cosec_user(
                            user_id=user_id,
                            ref_user_id=ref_user_id,
                            name=employee_name,
                            user_active=True,
                            validity_enable=True,
                            validity_date_dd=1,
                            validity_date_mm=1,
                            validity_date_yyyy=2035,
                        )
                        response = user.get("Response-Code")
                        if response and response == "0":
                            BiometricEmployees.objects.create(
                                ref_user_id=ref_user_id,
                                user_id=user_id,
                                employee_id=employee,
                                device_id=device,
                            )
        except Exception as error:
            if device.machine_type == "zk":
                conn.disable_device()
                logger.error(f"An error occurred: {error}")
        return HttpResponse("<script>window.location.reload()</script>")
    return render(
        request,
        "biometric/add_biometric_user.html",
        context={"form": employee_add_form, "device_id": device_id},
    )


@login_required
@install_required
@hx_request_required
def map_biometric_users(request, device_id):
    """
    Maps an horilla employee to a biometric user on a specified biometric device.
    """
    device = BiometricDevices.find(device_id)
    form = MapBioUsers(request.POST or None)
    template = "biometric_users/dahua/map_dahua_users.html"

    if device.machine_type == "etimeoffice":
        template = "biometric_users/etimeoffice/map_etimeoffice_users.html"
        form.fields["user_id"].label = _("Emp Code")

    if request.method == "POST" and form.is_valid():
        user_id = form.cleaned_data["user_id"]
        employee = form.cleaned_data["employee_id"]
        if device and employee:
            BiometricEmployees.objects.create(
                user_id=user_id, employee_id=employee, device_id=device
            )

            messages.success(
                request,
                _("Selected employee successfully mapped to the biometric user"),
            )
            form = MapBioUsers()

            if device.machine_type == "etimeoffice":
                form.fields["user_id"].label = _("Emp Code")

    return render(
        request,
        template,
        {"form": form, "device_id": device_id},
    )


@login_required
@install_required
@hx_request_required
def add_dahua_biometric_user(request, device_id):
    """
    Adds a new employee to a Dahua biometric device.

    This view handles the process of adding an employee as a user to a Dahua biometric device.
    """
    device = BiometricDevices.find(device_id)
    form = DahuaUserForm()
    if request.method == "POST":
        form = DahuaUserForm(request.POST)
        if form.is_valid():
            employee_id = request.POST.get("employee")
            card_no = request.POST.get("card_no")
            user_id = request.POST.get("user_id")
            card_status = request.POST.get("card_status")
            card_type = request.POST.get("card_type")
            password = request.POST.get("password")
            valid_date_end = request.POST.get("valid_date_end")

            try:
                employee = Employee.objects.get(id=employee_id) if employee_id else None
            except Employee.DoesNotExist:
                messages.error(request, _("Employee not found."))
                return render(
                    request, "biometric_users/dahua/add_dahua_user.html", context
                )

            dahua = DahuaAPI(
                ip=device.machine_ip,
                username=device.bio_username,
                password=device.bio_password,
            )

            response = dahua.enroll_new_user(
                card_name=employee.get_full_name() if employee else "",
                card_no=card_no,
                user_id=user_id,
                card_status=card_status,
                card_type=card_type,
                password=password,
                valid_date_end=valid_date_end,
            )

            if response.get("status_code") == 200:
                BiometricEmployees.objects.create(
                    dahua_card_no=card_no,
                    user_id=user_id,
                    employee_id=employee,
                    device_id=device,
                )
                messages.success(
                    request,
                    _("{} added to biometric device successfully").format(
                        employee.get_full_name() if employee else ""
                    ),
                )
                form = DahuaUserForm()
            else:
                messages.error(request, _("Failed to add user to biometric device."))
    context = {"form": form, "device_id": device_id}
    return render(request, "biometric_users/dahua/add_dahua_user.html", context)


@login_required
@hx_request_required
@install_required
def find_employee_badge_id(request):
    """
    Retrieves the badge ID of an employee based on their employee ID.
    """
    employee_id = request.GET.get("employee")
    user_id = Employee.objects.get(id=employee_id).badge_id if employee_id else ""
    input_field = f"""
    <input type="text" name="user_id" maxlength="50" class="oh-input w-100"
           placeholder="User ID" required="" id="id_user_id" value="{user_id}">
    """
    return HttpResponse(input_field)


@login_required
@hx_request_required
@install_required
def delete_dahua_user(request, obj_id=None):
    """
    Deletes a Dahua biometric user or multiple users from a device.
    """
    script = "<script>window.location.reload();</script>"
    try:
        if request.method == "POST" and obj_id:
            user = BiometricEmployees.objects.get(id=obj_id)
            user.delete()
            messages.success(
                request, _("{} successfully deleted!").format(user.employee_id)
            )
            script = "<script>reloadMessage();</script>"
        if request.method == "DELETE":
            user_ids = request.GET.getlist("ids")
            device_id = request.GET.get("device_id")
            if device_id:
                script = f"""
                            <span hx-get="/biometric/biometric-device-employees/{device_id}/"
                                hx-target="#dahuUsersList" hx-select="#dahuUsersList" hx-trigger="load delay:200ms"
                                hx-swap="outerHTML" hx-on-htmx-before-request="reloadMessage();">
                            </span>
                        """
            if user_ids:
                users = BiometricEmployees.objects.filter(user_id__in=user_ids)
                if users:
                    count = users.count()
                    users.delete()
                    messages.success(
                        request, _("{} users successfully deleted!").format(count)
                    ),
                else:
                    messages.warning(
                        request,
                        _("No rows are selected for deleting users from device."),
                    )
    except Exception as e:
        messages.error(request, _("An error occurred: {}").format(str(e)))
    return HttpResponse(script)


@login_required
@install_required
@hx_request_required
@permission_required("biometric.delete_biometricemployees")
def delete_etimeoffice_user(request, obj_id=None):
    """
    Deletes a user or multiple users from the eTimeOffice biometric system.
    """
    script = "<script>window.location.href = '/';</script>"
    if request.method == "POST":
        user = BiometricEmployees.objects.get(id=obj_id)
        device_id = user.device_id.id
        user.delete()
        messages.success(
            request, _("{} successfully deleted!").format(user.employee_id)
        )
        script = "<script>reloadMessage();</script>"
    if request.method == "DELETE":
        user_ids = request.GET.getlist("ids")
        device_id = request.GET.get("device_id")
        if device_id:
            script = f"""
                            <span hx-get="/biometric/biometric-device-employees/{device_id}/"
                                hx-target="#eTimeOfficeUsersList" hx-select="#eTimeOfficeUsersList" hx-trigger="load delay:200ms"
                                hx-swap="outerHTML" hx-on-htmx-before-request="reloadMessage();">
                            </span>
                        """
        if user_ids:
            users = BiometricEmployees.objects.filter(user_id__in=user_ids)
            if users:
                count = users.count()
                users.delete()
                messages.success(
                    request, _("{} users successfully deleted!").format(count)
                ),
            else:
                messages.warning(
                    request,
                    _("No rows are selected for deleting users from device."),
                )

    return HttpResponse(script)


@login_required
@install_required
@hx_request_required
@permission_required("biometric.change_biometricdevices")
def biometric_device_live(request):
    """
    Activate or deactivate live capture mode for a biometric device based on the request parameters.
    Now includes ZKBiotime Web API fallback support for ZK devices.

    :param request: The Django request object.
    :return: A JsonResponse containing a script to be executed on the client side.
    """
    logger.debug("biometric_device_live function called.")
    is_live = request.GET.get("is_live")
    device_id = request.GET.get("deviceId")
    logger.debug(f"Received request params - is_live: {is_live}, deviceId: {device_id}")

    try:
        device = BiometricDevices.objects.get(id=device_id)
        logger.debug(f"Biometric device retrieved: {device}")
    except BiometricDevices.DoesNotExist:
        logger.error(f"Device with id {device_id} does not exist.")
        return HttpResponse("""
        <script>
            Swal.fire({
              title : "Device not found",
              text: "The selected biometric device could not be found.",
              icon: "error",
              showConfirmButton: true,
            });
        </script>
        """)

    is_live = is_live == "on"
    conn = None

    if is_live:
        port_no = device.port
        machine_ip = device.machine_ip
        password = int(device.zk_password)

        try:
            if device.machine_type == "zk":
                logger.debug("Initializing ZK biometric device with API fallback support.")
                
                # First try direct ZK connection
                try:
                    zk_device = ZK(
                        machine_ip,
                        port=port_no,
                        timeout=5,
                        password=int(password),
                        force_udp=False,
                        ommit_ping=False,
                    )
                    conn = zk_device.connect()
                    if conn:
                        conn.test_voice(index=14)
                        logger.info("Direct ZK device connection successful.")
                        connection_method = "direct"
                    else:
                        raise Exception("Failed to establish direct ZK connection")
                        
                except Exception as direct_error:
                    logger.warning(f"Direct ZK connection failed: {direct_error}")
                    
                    # Try ZKBiotime API fallback
                    hardcoded_api_url = ""  # TODO: Replace with device configuration
                    hardcoded_username = ""  # TODO: Replace with device configuration  
                    hardcoded_password = ""  # TODO: Replace with device configuration
                    
                    logger.info("Attempting ZKBiotime API fallback connection...")
                    
                    try:
                        # Test ZKBiotime API connection
                        if test_zkbiotime_connection(hardcoded_api_url, hardcoded_username, hardcoded_password):
                            logger.info("ZKBiotime API fallback connection successful.")
                            connection_method = "api"
                            conn = None  # No direct connection available
                        else:
                            raise Exception("ZKBiotime API connection test failed")
                            
                    except Exception as api_error:
                        logger.error(f"Both direct ZK and API fallback failed: {api_error}")
                        raise Exception("All connection methods failed")

                # Start the attendance thread with fallback support
                instance = ZKBioAttendance(machine_ip, port_no, password)
                
                # Set the API fallback mode based on connection method
                if connection_method == "api":
                    instance.api_fallback_active = True
                    logger.info("Starting ZKBioAttendance in API fallback mode")
                else:
                    instance.api_fallback_active = False
                    logger.info("Starting ZKBioAttendance in direct connection mode")
                
                # Update device status
                device.is_live = True
                device.is_scheduler = False
                device.save()
                updated_device = BiometricDevices.objects.get(id=device.id)
                logger.debug(f"Post-save check: device.is_live = {updated_device.is_live}")
                
                # Start the thread
                instance.start()
                BIO_DEVICE_THREADS[device.id] = instance
                
                if connection_method == "api":
                    logger.info(f"Live mode activated for ZK device {device_id} using ZKBiotime API fallback")
                    script = """<script>
                            Swal.fire({
                              text: "Live capture activated using ZKBiotime API (fallback mode).",
                              icon: "success",
                              showConfirmButton: false,
                              timer: 2000,
                              timerProgressBar: true,
                              didClose: () => {
                                location.reload();
                                },
                            });
                            </script>
                        """
                else:
                    logger.info(f"Live mode activated for ZK device {device_id} using direct connection")
                    script = """<script>
                            Swal.fire({
                              text: "Live capture activated using direct device connection.",
                              icon: "success",
                              showConfirmButton: false,
                              timer: 1500,
                              timerProgressBar: true,
                              didClose: () => {
                                location.reload();
                                },
                            });
                            </script>
                        """
                        
            elif device.machine_type == "cosec":
                logger.debug("Initializing COSEC biometric device.")
                cosec = COSECBiometric(
                    device.machine_ip,
                    device.port,
                    device.bio_username,
                    device.bio_password,
                    timeout=10,
                )
                response = cosec.basic_config()
                logger.debug(f"COSEC config response: {response}")
                if response.get("app"):
                    device.is_live = True
                    device.is_scheduler = False
                    device.save()
                    thread = COSECBioAttendanceThread(device.id)
                    thread.start()
                    BIO_DEVICE_THREADS[device.id] = thread
                    logger.info(f"Live mode activated for COSEC device {device_id}")
                else:
                    logger.warning("COSEC config failed. TimeoutError raised.")
                    raise TimeoutError
                    
                script = """<script>
                        Swal.fire({
                          text: "The live capture mode has been activated successfully.",
                          icon: "success",
                          showConfirmButton: false,
                          timer: 1500,
                          timerProgressBar: true,
                          didClose: () => {
                            location.reload();
                            },
                        });
                        </script>
                    """
            else:
                logger.warning(f"Unsupported machine type: {device.machine_type}")
                script = """<script>
                        Swal.fire({
                          text: "The live capture mode has been activated successfully.",
                          icon: "success",
                          showConfirmButton: false,
                          timer: 1500,
                          timerProgressBar: true,
                          didClose: () => {
                            location.reload();
                            },
                        });
                        </script>
                    """

        except TimeoutError as error:
            logger.error(f"TimeoutError for device {device_id}: {str(error)}")
            device.is_live = False
            device.save()
            script = """
           <script>
                Swal.fire({
                  title : "Connection unsuccessful",
                  text: "Please double-check the accuracy of the provided IP Address and Port Number for correctness",
                  icon: "warning",
                  showConfirmButton: false,
                  timer: 3000,
                  timerProgressBar: true,
                  didClose: () => {
                    location.reload();
                    },
                });
            </script>
            """
        except Exception as e:
            logger.error(f"Error activating live mode for device {device_id}: {str(e)}")
            device.is_live = False
            device.save()
            
            # Provide specific error messages based on error type
            if "All connection methods failed" in str(e):
                script = """
                <script>
                    Swal.fire({
                      title: "Connection Failed",
                      text: "Both direct device connection and ZKBiotime API failed. Please check device settings.",
                      icon: "error",
                      showConfirmButton: true,
                    });
                </script>
                """
            else:
                script = """
                <script>
                    Swal.fire({
                      title: "Unexpected Error",
                      text: "Something went wrong while activating live mode. Please try again.",
                      icon: "error",
                      showConfirmButton: true,
                    });
                </script>
                """
        finally:
            if conn:
                try:
                    conn.disconnect()
                    logger.debug("ZK device connection closed.")
                except Exception as disconnect_error:
                    logger.warning(f"Error while disconnecting device {device_id}: {disconnect_error}")
    else:
        # Deactivate live mode
        try:
            device.is_live = False
            device.save()
            logger.info(f"Live mode deactivated for device {device_id}")

            # Stop existing threads based on device type
            existing_thread = BIO_DEVICE_THREADS.get(device.id)
            if existing_thread:
                existing_thread.stop()
                del BIO_DEVICE_THREADS[device.id]
                logger.debug(f"Stopped and removed thread for device {device_id}")
                
        except Exception as e:
            logger.exception(f"Error during live deactivation for device {device_id}: {str(e)}")

        script = """
           <script>
                Swal.fire({
                  text: "The live capture mode has been deactivated successfully.",
                  icon: "warning",
                  showConfirmButton: false,
                  timer: 3000,
                  timerProgressBar: true,
                  didClose: () => {
                    location.reload();
                    },
                });
            </script>
            """

    return HttpResponse(script)

def zk_biometric_attendance_logs(device):
    """
    Retrieve attendance records from a ZK biometric device and update the clock-in/clock-out status.

    :param device: The BiometricDevices instance representing the ZK device.
    """
    logger.debug("Starting zk_biometric_attendance_logs for device %s (IP: %s, Port: %s)", 
                 device.id, device.machine_ip, device.port)

    port_no = device.port
    machine_ip = device.machine_ip
    conn = None
    zk_device = ZK(
        machine_ip,
        port=port_no,
        timeout=5,
        password=int(device.zk_password),
        force_udp=False,
        ommit_ping=False,
    )
    try:
        conn = zk_device.connect()
        conn.enable_device()
        logger.info("Connected to ZK device at %s:%s", machine_ip, port_no)

        attendances = conn.get_attendance()
        logger.debug("Retrieved %d attendance records from device %s", len(attendances), device.id)

        # Determine the cutoff timestamp for filtering
        last_attendance_datetime = attendances[-1].timestamp if attendances else None
        if device.last_fetch_date and device.last_fetch_time:
            filtered_attendances = [
                a for a in attendances
                if (a.timestamp.date() > device.last_fetch_date)
                or (
                    a.timestamp.date() == device.last_fetch_date
                    and a.timestamp.time() > device.last_fetch_time
                )
            ]
            logger.debug(
                "Filtered attendances: %d new records since %s %s", 
                len(filtered_attendances), device.last_fetch_date, device.last_fetch_time
            )
        else:
            filtered_attendances = attendances
            logger.debug("No previous fetch date/time set; processing all %d records", len(filtered_attendances))

        # Update device last fetch markers
        if last_attendance_datetime:
            device.last_fetch_date = last_attendance_datetime.date()
            device.last_fetch_time = last_attendance_datetime.time()
            device.save()
            logger.info(
                "Updated device %s last_fetch_date to %s and last_fetch_time to %s",
                device.id, device.last_fetch_date, device.last_fetch_time
            )

        # Map user IDs to BiometricEmployees entries
        bio_id_map = {
            bio.user_id: bio
            for bio in BiometricEmployees.objects.filter(device_id=device)
        }
        logger.debug("Built biometric ID map with %d entries", len(bio_id_map))

        # Process each attendance record
        for attendance in filtered_attendances:
            user_id = attendance.user_id
            punch_code = attendance.punch
            date_time = django_timezone.make_aware(attendance.timestamp)
            date = date_time.date()
            time = date_time.time()

            logger.debug(
                "Processing attendance: device=%s, user=%s, timestamp=%s, punch=%s",
                device.id, user_id, date_time, punch_code
            )

            bio_id = bio_id_map.get(user_id)
            if not bio_id:
                logger.warning(
                    "User ID %s not found in BiometricEmployees for device %s", 
                    user_id, device.id
                )
                continue

            request_data = Request(
                user=bio_id.employee_id.employee_user_id,
                date=date,
                time=time,
                datetime=date_time,
            )

            # Determine clock-in or clock-out
            if punch_code in {0, 3, 4}:
                logger.info("Clocking in user %s at %s", user_id, date_time)
                try:
                    clock_in(request_data)
                except Exception as error:
                    logger.error(
                        "Error clocking in user %s at %s: %s", 
                        user_id, date_time, error, exc_info=True
                    )
            elif punch_code in {1, 2, 5}:
                logger.info("Clocking out user %s at %s", user_id, date_time)
                try:
                    clock_out(request_data)
                except Exception as error:
                    logger.error(
                        "Error clocking out user %s at %s: %s", 
                        user_id, date_time, error, exc_info=True
                    )
            else:
                logger.debug(
                    "Skipping attendance for user %s with unrecognized punch code %s", 
                    user_id, punch_code
                )

        return len(filtered_attendances), None

    except zk_exception.ZKErrorResponse as e:
        logger.error(
            "ZK device error response for device %s: %s", 
            device.id, e, exc_info=True
        )
        return "error", str(e)
    except Exception as e:
        logger.error(
            "Unexpected error in zk_biometric_attendance_logs for device %s: %s", 
            device.id, e, exc_info=True
        )
        return "error", str(e)
    finally:
        if conn:
            conn.disconnect()
            logger.info("Disconnected from ZK device %s", device.id)


def zk_biometric_attendance_scheduler(device_id):
    """
    Scheduler function used for attendance logs
    """
    logger.debug("Scheduler triggered for device_id %s", device_id)
    device = BiometricDevices.find(device_id)
    if not device:
        logger.warning("Device with ID %s not found", device_id)
        return

    if device.is_scheduler:
        logger.info("Running attendance log retrieval for device %s", device_id)
        zk_biometric_attendance_logs(device)
    else:
        logger.info("Device %s is not enabled for scheduler; skipping", device_id)


def anviz_biometric_attendance_logs(device):
    """
    Retrieves attendance records from an Anviz biometric device and processes them.

    :param device_id: The Object Id of the Anviz biometric device.
    """
    current_utc_time = datetime.utcnow()
    anviz_device = CrossChexCloudAPI(
        api_url=device.api_url,
        api_key=device.api_key,
        api_secret=device.api_secret,
        anviz_request_id=device.anviz_request_id,
    )
    begin_time = (
        datetime.combine(device.last_fetch_date, device.last_fetch_time)
        if device.last_fetch_date and device.last_fetch_time
        else current_utc_time.replace(hour=0, minute=0, second=0, microsecond=0)
    )
    attendance_records = anviz_device.get_attendance_records(
        begin_time=begin_time, token=device.api_token
    )
    device.last_fetch_date, device.last_fetch_time = (
        current_utc_time.date(),
        current_utc_time.time(),
    )
    device.save()
    for attendance in attendance_records["list"]:
        badge_id = attendance["employee"]["workno"]
        punch_code = attendance["checktype"]
        date_time_utc = datetime.strptime(
            attendance["checktime"], "%Y-%m-%dT%H:%M:%S%z"
        )
        date_time_obj = date_time_utc.astimezone(django_timezone.get_current_timezone())
        employee = Employee.objects.filter(badge_id=badge_id).first()
        if employee:
            request_data = Request(
                user=employee.employee_user_id,
                date=date_time_obj.date(),
                time=date_time_obj.time(),
                datetime=date_time_obj,
            )
            if punch_code in {0, 128}:
                try:
                    clock_in(request_data)
                except Exception as error:
                    logger.error("Error in clock in ", error)
            else:
                try:
                    # // 1 , 129 check type check out and door close
                    clock_out(request_data)
                except Exception as error:
                    logger.error("Error in clock out ", error)
    return len(attendance_records["list"])


def anviz_biometric_attendance_scheduler(device_id):
    """
    Schedules the attendance log retrieval for an Anviz biometric device.
    """
    device = BiometricDevices.find(device_id)
    if device and device.is_scheduler:
        anviz_biometric_attendance_logs(device)


def cosec_biometric_attendance_logs(device):
    """
    Retrieves and processes attendance logs from a COSEC biometric device.
    """
    device_args = COSECAttendanceArguments.objects.filter(device_id=device).first()
    last_fetch_roll_ovr_count = (
        int(device_args.last_fetch_roll_ovr_count) if device_args else 0
    )
    last_fetch_seq_number = int(device_args.last_fetch_seq_number) if device_args else 1

    cosec = COSECBiometric(
        device.machine_ip,
        device.port,
        device.bio_username,
        device.bio_password,
        timeout=10,
    )
    attendances = cosec.get_attendance_events(
        last_fetch_roll_ovr_count, int(last_fetch_seq_number) + 1
    )

    if not isinstance(attendances, list):
        return

    for attendance in attendances:
        ref_user_id = attendance["detail-1"]
        employee = BiometricEmployees.objects.filter(ref_user_id=ref_user_id).first()
        if not employee:
            continue

        date_str = attendance["date"]
        time_str = attendance["time"]
        attendance_date = datetime.strptime(date_str, "%d/%m/%Y").date()
        attendance_time = datetime.strptime(time_str, "%H:%M:%S").time()
        attendance_datetime = datetime.combine(attendance_date, attendance_time)
        punch_code = attendance["detail-2"]

        request_data = Request(
            user=employee.employee_id.employee_user_id,
            date=attendance_date,
            time=attendance_time,
            datetime=django_timezone.make_aware(attendance_datetime),
        )

        try:
            if punch_code in ["1", "3", "5", "7", "9", "0"]:
                clock_in(request_data)
            elif punch_code in ["2", "4", "6", "8", "10"]:
                clock_out(request_data)
            else:
                pass
        except Exception as error:
            logger.error("Error processing attendance: ", error)

    if attendances:
        last_attendance = attendances[-1]
        COSECAttendanceArguments.objects.update_or_create(
            device_id=device,
            defaults={
                "last_fetch_roll_ovr_count": last_attendance["roll-over-count"],
                "last_fetch_seq_number": last_attendance["seq-No"],
            },
        )
    return len(attendances)


def cosec_biometric_attendance_scheduler(device_id):
    """
    Retrieve and process attendance events from a COSEC biometric device.

    This function fetches attendance events from the specified COSEC biometric device
    and processes them to record clock-in and clock-out events for employees.

    Args:
        device_id (uuid): The ID of the COSEC biometric device.
    """
    device = BiometricDevices.find(device_id)
    if device and device.is_scheduler:
        cosec_biometric_attendance_logs(device)


def dahua_biometric_attendance_logs(device):
    """
    Retrieves logs from a Dahua biometric device and marks attendance in Horilla.

    This function fetches biometric logs from the specified device, processes the attendance records,
    and updates the attendance system in Horilla. If an employee has an active clock-in record,
    it marks their clock-out; otherwise, it registers a new clock-in entry.

    Args:
        device_id (int): The unique identifier of the biometric device.

    Returns:
        None
    """
    begin_time = (
        datetime.combine(device.last_fetch_date, device.last_fetch_time)
        + timedelta(seconds=1)
        if device.last_fetch_date and device.last_fetch_time
        else datetime.combine(datetime.today(), datetime.min.time())
    )

    dahua = DahuaAPI(
        ip=device.machine_ip, username=device.bio_username, password=device.bio_password
    )
    logs = dahua.get_control_card_rec(start_time=begin_time)

    if logs.get("status_code") == 200:
        for log in logs.get("records", []):
            user_id = log.get("user_id")
            if not user_id:
                continue

            employee = BiometricEmployees.objects.filter(
                user_id=user_id, device_id=device
            ).first()
            if not employee:
                continue

            attendance_datetime = log.get("create_time")
            user_tz = pytz.timezone(TIME_ZONE)
            attendance_datetime = attendance_datetime.astimezone(user_tz)

            last_none_activity = (
                AttendanceActivity.objects.filter(
                    employee_id=employee.employee_id,
                    clock_out=None,
                )
                .order_by("in_datetime")
                .last()
            )

            request_data = Request(
                user=employee.employee_id.employee_user_id,
                date=attendance_datetime.date(),
                time=attendance_datetime.time(),
                datetime=attendance_datetime,
            )

            if last_none_activity:
                clock_out(request_data)
            else:
                clock_in(request_data)

        if logs.get("records"):
            last_log = logs["records"][-1]
            device.last_fetch_date = last_log["create_time"].date()
            device.last_fetch_time = last_log["create_time"].time()
            device.save()
        return len(logs.get("records", []))
    else:
        return "error"


def dahua_biometric_attendance_scheduler(device_id):
    """
    Schedules the attendance log retrieval for a Dahua biometric device.
    """
    device = BiometricDevices.find(device_id)
    if device and device.is_scheduler:
        dahua_biometric_attendance_logs(device)


def etimeoffice_biometric_attendance_logs(device):
    """
    Retrieves and processes attendance logs from an eTimeOffice biometric device.
    """
    now = datetime.now()
    etimeoffice = ETimeOfficeAPI(
        username=device.bio_username,
        password=device.bio_password,
    )

    from_date = (
        f"{(datetime.combine(device.last_fetch_date, device.last_fetch_time) + timedelta(minutes=1)):%d/%m/%Y_%H:%M}"
        if device.last_fetch_date and device.last_fetch_time
        else f"{now:%d/%m/%Y}_00:00"
    )
    to_date = f"{now:%d/%m/%Y_%H:%M}"

    logs = etimeoffice.download_punch_data(from_date=from_date, to_date=to_date)
    if logs.get("Msg") != "Success":
        return "error"

    punch_data = logs.get("PunchData", [])
    if not punch_data:
        return 0

    user_tz = pytz.timezone(TIME_ZONE)

    employee_map = {
        emp.user_id: emp for emp in BiometricEmployees.objects.filter(device_id=device)
    }

    for log in reversed(punch_data):
        user_id = log.get("Empcode")
        if not user_id or user_id not in employee_map:
            continue

        employee = employee_map[user_id]
        attendance_datetime = log["PunchDate"].astimezone(user_tz)

        request_data = Request(
            user=employee.employee_id.employee_user_id,
            date=attendance_datetime.date(),
            time=attendance_datetime.time(),
            datetime=attendance_datetime,
        )

        last_none_activity = (
            AttendanceActivity.objects.filter(
                employee_id=employee.employee_id,
                clock_out=None,
            )
            .order_by("in_datetime")
            .last()
        )

        if last_none_activity:
            clock_out(request_data)
        else:
            clock_in(request_data)

    last_log = punch_data[0]
    device.last_fetch_date, device.last_fetch_time = (
        last_log["PunchDate"].date(),
        last_log["PunchDate"].time(),
    )
    device.save()

    return len(punch_data)


def etimeoffice_biometric_attendance_scheduler(device_id):
    """
    Schedules the attendance log retrieval for an eTimeOffice biometric device.
    """
    device = BiometricDevices.find(device_id)
    if device and device.is_scheduler:
        etimeoffice_biometric_attendance_logs(device)


try:
    devices = BiometricDevices.objects.all().update(is_live=False)
    for device in BiometricDevices.objects.filter(is_scheduler=True):
        if device:
            if str_time_seconds(device.scheduler_duration) > 0:
                if device.machine_type == "anviz":
                    scheduler = BackgroundScheduler()
                    scheduler.add_job(
                        lambda: anviz_biometric_attendance_scheduler(device.id),
                        "interval",
                        seconds=str_time_seconds(device.scheduler_duration),
                    )
                    scheduler.start()
                elif device.machine_type == "zk":
                    scheduler = BackgroundScheduler()
                    scheduler.add_job(
                        lambda: zk_biometric_attendance_scheduler(device.id),
                        "interval",
                        seconds=str_time_seconds(device.scheduler_duration),
                        id=f"biometric_{device.id}",
                    )
                    scheduler.start()
                elif device.machine_type == "dahua":
                    scheduler = BackgroundScheduler()
                    scheduler.add_job(
                        lambda: dahua_biometric_attendance_scheduler(device.id),
                        "interval",
                        seconds=str_time_seconds(device.scheduler_duration),
                    )
                    scheduler.start()

                elif device.machine_type == "cosec":
                    scheduler = BackgroundScheduler()
                    scheduler.add_job(
                        lambda: cosec_biometric_attendance_scheduler(device.id),
                        "interval",
                        seconds=str_time_seconds(device.scheduler_duration),
                    )
                    scheduler.start()

                elif device.machine_type == "etimeoffice":
                    scheduler = BackgroundScheduler()
                    scheduler.add_job(
                        lambda: etimeoffice_biometric_attendance_scheduler(device.id),
                        "interval",
                        seconds=str_time_seconds(device.scheduler_duration),
                    )
                    scheduler.start()
                else:
                    pass
except:
    pass


### TD_HRMS_CODE_START_HERE
# logging.basicConfig(level=logging.DEBUG)  # Set logging level to DEBUG

@login_required
@install_required
def sync_zk_users(request, device_id):
    """
    Fetches all users from a ZK device and displays them for mapping.
    Now includes ZKBiotime Web API fallback support.
    """
    logger.info(f"Starting sync_zk_users for device_id: {device_id}")
    conn = None
    
    try:
        device = BiometricDevices.find(device_id)
        if not device:
            logger.error(f"Device {device_id} not found")
            messages.error(request, _("Device not found"))
            return redirect("/biometric/view-biometric-devices/")
            
        if device.machine_type != "zk":
            logger.error(f"Device {device_id} is not a ZK device")
            messages.error(request, _("This function is only for ZK devices"))
            return redirect(f"/biometric/biometric-device-employees/{device_id}/")

        zk_users = []
        
        # First try direct ZK connection
        try:
            logger.debug(f"Attempting direct connection to ZK device at {device.machine_ip}:{device.port}")
            zk_device = ZK(
                device.machine_ip,
                port=device.port,
                timeout=5,
                password=device.zk_password,
                force_udp=False,
                ommit_ping=False,
            )
            
            try:
                conn = zk_device.connect()
                if conn:
                    logger.info("Successfully connected to ZK device")
                    try:
                        zk_users = conn.get_users()
                        logger.debug(f"Fetched {len(zk_users)} users from ZK device")
                    except Exception as user_error:
                        logger.error(f"Error fetching users from ZK device: {user_error}")
                        raise Exception("Failed to fetch users from device")
                    finally:
                        try:
                            conn.disconnect()
                            logger.debug("Disconnected from ZK device")
                        except Exception as disconnect_error:
                            logger.warning(f"Error disconnecting: {disconnect_error}")
                else:
                    logger.warning("Direct ZK connection failed, trying API fallback")
                    raise Exception("Failed to establish direct connection")
                    
            except Exception as conn_error:
                logger.error(f"Connection error: {conn_error}")
                raise Exception("Connection failed")

        except Exception as direct_error:
            logger.warning(f"Direct ZK access failed: {direct_error}, trying API fallback")
            
            # Try ZKBiotime API fallback
            try:
                # Hardcoded API configuration (same as in ZKBioAttendance)
                hardcoded_api_url = ""  # TODO: Replace with device configuration
                hardcoded_username = ""
                hardcoded_password = ""
                
                # Get API token
                token = get_zkbiotime_token(
                    hardcoded_api_url,
                    hardcoded_username,
                    hardcoded_password
                )
                
                if token:
                    logger.info("Successfully got ZKBiotime API token")
                    
                    # Fetch employees from API
                    headers = {
                        "Authorization": f"Token {token}",
                        "Content-Type": "application/json"
                    }
                    
                    try:
                        employees_url = f"{hardcoded_api_url}/personnel/api/employees/"
                        response = requests.get(
                            employees_url, 
                            headers=headers,
                            params={"page_size": 100},
                            timeout=10
                        )
                        
                        if response.status_code == 200:
                            api_data = response.json()
                            
                            # Convert API data to format similar to ZK users
                            class ApiUser:
                                def __init__(self, uid, user_id, name):
                                    self.uid = uid
                                    self.user_id = user_id
                                    self.name = name
                                    self.privilege = 0
                                    self.password = ""
                                    self.group_id = ""
                            
                            zk_users = []
                            for emp in api_data.get("data", []):
                                user = ApiUser(
                                    uid=emp.get("id"),
                                    user_id=emp.get("emp_code"),
                                    name=f"{emp.get('first_name', '')} {emp.get('last_name', '')}".strip()
                                )
                                zk_users.append(user)
                                
                            logger.info(f"Retrieved {len(zk_users)} users from ZKBiotime API")
                        else:
                            logger.error(f"API request failed: {response.status_code}")
                            messages.error(request, _("Failed to fetch users from ZKBiotime API"))
                            
                    except requests.exceptions.RequestException as api_error:
                        logger.error(f"API request error: {api_error}")
                        messages.error(request, _("Error connecting to ZKBiotime API"))
                else:
                    logger.error("Failed to get ZKBiotime API token")
                    messages.error(request, _("Authentication failed with ZKBiotime API"))
                    
            except Exception as api_error:
                logger.error(f"ZKBiotime API fallback failed: {api_error}")
                messages.error(request, _("Both direct connection and API fallback failed"))
                return redirect(f"/biometric/biometric-device-employees/{device_id}/")

        # Get all active employees for mapping
        try:
            employees = Employee.objects.filter(is_active=True)
            logger.debug(f"Fetched {employees.count()} active employees from database")
            
            context = {
                "zk_users": zk_users,
                "employees": employees,
                "device_id": device_id,
                "is_api_mode": not bool(conn)  # Indicate if using API mode
            }
            
            logger.info("Rendering sync_zk_users template")
            return render(request, "biometric_users/zk/sync_zk_users.html", context)
            
        except Exception as db_error:
            logger.error(f"Database error: {db_error}")
            messages.error(request, _("Error fetching employee data"))
            
    except Exception as e:
        logger.exception(f"Unexpected error in sync_zk_users: {e}")
        messages.error(request, _(f"Error: {str(e)}"))

    return redirect(f"/biometric/biometric-device-employees/{device_id}/")

@login_required
@install_required
def map_zk_users(request, device_id):
    """
    Maps ZK users to Horilla employees based on the submitted form.
    """
    logger.info(f"Starting map_zk_users for device_id: {device_id}")
    if request.method == "POST":
        device = BiometricDevices.find(device_id)
        if device.machine_type != "zk":
            logger.error(f"Device {device_id} is not a ZK device.")
            messages.error(request, _("This function is only for ZK devices"))
            return redirect(f"/biometric/biometric-device-employees/{device_id}/")

        try:
            mappings = []
            logger.debug("Processing submitted form data for mapping.")
            for key, value in request.POST.items():
                if key.startswith("employee_mapping_") and value:
                    user_id = key.replace("employee_mapping_", "")
                    employee_id = value
                    logger.debug(f"Mapping user_id: {user_id} to employee_id: {employee_id}")
                    mappings.append(
                        BiometricEmployees(
                            user_id=user_id,
                            employee_id_id=employee_id,
                            device_id=device,
                        )
                    )

            # Bulk create mappings
            BiometricEmployees.objects.bulk_create(mappings)
            logger.info(f"Successfully mapped {len(mappings)} users to employees.")
            messages.success(request, _("Users successfully mapped to employees"))
        except Exception as e:
            logger.exception(f"Error occurred while mapping ZK users: {e}")
            messages.error(request, _(f"Error: {e}"))

    return redirect(f"/biometric/biometric-device-employees/{device_id}/")


@login_required
@install_required
@hx_request_required
def map_biometric_users_zk(request, device_id):
    """
    Maps an horilla employee to a biometric user on a specified biometric device.
    Now includes ZKBiotime Web API fallback for user mapping.
    """
    logger.info(f"Starting map_biometric_users_zk for device_id: {device_id}")
    device = BiometricDevices.find(device_id)
    form = MapBioUsers(request.POST or None)
    zk_users = []

    # Select template based on device type
    if device.machine_type == "zk":
        template = "biometric_users/zk/map_zk_users.html"
        logger.debug(f"Using template: {template}")

        # First try direct ZK connection
        try:
            logger.debug(f"Attempting direct connection to ZK device at {device.machine_ip}:{device.port}")
            zk_device = ZK(
                device.machine_ip,
                port=device.port,
                timeout=5,
                password=device.zk_password,
                force_udp=False,
                ommit_ping=False,
            )
            
            try:
                conn = zk_device.connect()
                if conn:
                    logger.info("Successfully connected to ZK device.")
                    try:
                        zk_users = conn.get_users()
                        logger.debug(f"Fetched {len(zk_users)} users from ZK device")
                    except Exception as user_error:
                        logger.error(f"Error fetching users from ZK device: {user_error}")
                        raise Exception("Failed to fetch users from device")
                    finally:
                        conn.disconnect()
                else:
                    logger.warning("Direct ZK connection failed, trying API fallback")
                    raise Exception("Failed to establish direct connection")
            except Exception as conn_error:
                logger.error(f"Connection error: {conn_error}")
                raise Exception("Connection failed")

        except Exception as direct_error:
            logger.warning(f"Direct ZK access failed: {direct_error}, trying ZKBiotime API fallback")
            
            # Try ZKBiotime API fallback
            try:
                # Hardcoded API credentials (same as in ZKBioAttendance)
                hardcoded_api_url = ""  # TODO: Replace with device configuration
                hardcoded_username = ""  # TODO: Replace with device configuration
                hardcoded_password = ""  # TODO: Replace with device configuration
                
                # Get API token
                token = get_zkbiotime_token(
                    hardcoded_api_url,
                    hardcoded_username,
                    hardcoded_password
                )
                
                if token:
                    logger.info("Successfully got ZKBiotime API token")
                    
                    # Fetch employees from API
                    headers = {
                        "Authorization": f"Token {token}",
                        "Content-Type": "application/json"
                    }
                    
                    try:
                        employees_url = f"{hardcoded_api_url}/personnel/api/employees/"
                        response = requests.get(
                            employees_url, 
                            headers=headers,
                            params={"page_size": 100},
                            timeout=10
                        )
                        
                        if response.status_code == 200:
                            api_data = response.json()
                            
                            # Convert API data to format similar to ZK device users
                            class ApiUser:
                                def __init__(self, user_id, name):
                                    self.user_id = user_id
                                    self.name = name
                            
                            zk_users = []
                            for emp in api_data.get("data", []):
                                user = ApiUser(
                                    user_id=emp.get("emp_code"),
                                    name=f"{emp.get('first_name', '')} {emp.get('last_name', '')}".strip()
                                )
                                zk_users.append(user)
                                
                            logger.info(f"Retrieved {len(zk_users)} users from ZKBiotime API")
                        else:
                            logger.error(f"API request failed: {response.status_code}")
                            messages.error(request, _("Failed to fetch users from ZKBiotime API"))
                            
                    except requests.exceptions.RequestException as api_error:
                        logger.error(f"API request error: {api_error}")
                        messages.error(request, _("Error connecting to ZKBiotime API"))
                else:
                    logger.error("Failed to get ZKBiotime API token")
                    messages.error(request, _("Authentication failed with ZKBiotime API"))
                    
            except Exception as api_error:
                logger.error(f"ZKBiotime API fallback failed: {api_error}")
                messages.error(request, _("Both direct connection and API fallback failed"))

    elif device.machine_type == "etimeoffice":
        template = "biometric_users/etimeoffice/map_etimeoffice_users.html"
        form.fields["user_id"].label = _("Emp Code")
        logger.debug(f"Using template: {template}")
    else:
        template = "biometric_users/dahua/map_dahua_users.html"
        logger.debug(f"Using template: {template}")

    if request.method == "POST" and form.is_valid():
        user_id = form.cleaned_data["user_id"]
        employee = form.cleaned_data["employee_id"]
        logger.debug(f"Form submitted with user_id: {user_id}, employee_id: {employee}")
        
        if device and employee:
            # For ZK devices, verify user_id exists
            if device.machine_type == "zk":
                user_exists = False
                
                # Check in local zk_users list first
                if zk_users:
                    user_exists = any(str(user.user_id) == str(user_id) for user in zk_users)
                
                # If not found and using API, verify through API
                if not user_exists and not zk_users:
                    try:
                        # Try API verification
                        token = get_zkbiotime_token(
                            hardcoded_api_url,
                            hardcoded_username,
                            hardcoded_password
                        )
                        
                        if token:
                            headers = {
                                "Authorization": f"Token {token}",
                                "Content-Type": "application/json"
                            }
                            
                            verify_url = f"{hardcoded_api_url}/personnel/api/employees/{user_id}/"
                            response = requests.get(verify_url, headers=headers, timeout=10)
                            user_exists = response.status_code == 200
                    except Exception as verify_error:
                        logger.error(f"Error verifying user through API: {verify_error}")
                        # Continue with user_exists = False
                
                if user_exists:
                    try:
                        BiometricEmployees.objects.create(
                            user_id=user_id,
                            employee_id=employee,
                            device_id=device,
                        )
                        logger.info(f"Successfully mapped user_id {user_id} to employee {employee.id}")
                        messages.success(request, _("Employee successfully mapped to biometric user"))
                    except Exception as create_error:
                        logger.error(f"Error creating mapping: {create_error}")
                        messages.error(request, _("Error creating employee mapping"))
                else:
                    logger.warning(f"User ID {user_id} not found in device or API")
                    messages.error(request, _("User ID not found in biometric system"))
            else:
                # Non-ZK device mapping
                try:
                    BiometricEmployees.objects.create(
                        user_id=user_id,
                        employee_id=employee,
                        device_id=device,
                    )
                    messages.success(request, _("Employee successfully mapped to biometric user"))
                except Exception as create_error:
                    logger.error(f"Error creating mapping: {create_error}")
                    messages.error(request, _("Error creating employee mapping"))
            
            form = MapBioUsers()
            if device.machine_type == "etimeoffice":
                form.fields["user_id"].label = _("Emp Code")

    context = {
        "form": form,
        "device_id": device_id,
        "zk_users": zk_users  # Pass users to template for display
    }
    
    logger.info("Rendering template with context")
    return render(request, template, context)

[mubashir-rehman]

Also, thread wasn't being saved to be stopped later on when live capture is enabled.
One more thing, I have created a view to map biometric users from device to HRMS users.
It can improved by adding a field for biometric device user id in employee model which can be used by biometric module and can be edit by admin or HR.

[mubashir-rehman]

My thoughts are all over the places. Let's communicate in detail where needed.

[mubashir-rehman]

@horilla-opensource Is that something that can be added to Horilla?

[horilla-opensource]

Hi @mubashir-rehman,

Thank you for sharing the details — your input highlights some very valid edge cases and technical constraints when dealing with biometric data syncing via ZKBioTime.

We’d be happy to review and test the integration on our end. If you could please share:

• ZKBioTime Integration functions code (even in its current state),
• A sample set of logs/screenshots showing the inconsistent check-in/out scenarios,
• And temporary ZKBioTime credentials (with limited access, if preferred),

…it would help us better understand the flow and simulate the issue accurately.

Looking forward to your input so we can move this forward collaboratively.

Best regards,
Horilla Team

[mubashir-rehman]

Thanks for your positive responce. ZKBioTime is a fallback mechanism we added to test against pyzk limitation. The same cases of attendance mismatch also happen in pyzk implementation of Horilla.
Following are the steps to reproduce:

1. Mark check in and check outs
2. Skip some check outs and check ins (such that record exists in biometric device but not in Horilla, let's assume there was some issue)
3. modify date of last fetch to previous date for BiometricDevice model
4. Fetch data again (in hope to add missing check ins and check outs)
5. Now the activities don't get assigned to date of attendance, and work records can become negative hours as well as positive hours.
   Let me get back to you with specific functions and if I am allowed with logs.
   I have shared the whole file where I made changes. However, I understand that it is not the correct way to share the code.