v0.229.019 bug fixes upgrade to v0.229.058

RELEASE_NOTES.md

@@ -1,11 +1,84 @@
 <!-- BEGIN RELEASE_NOTES.MD BLOCK -->
 # Feature Release
 
-### **(v0.229.014)**
+### **(v0.229.058)**
+
+#### New Features
+
+*   **Admin Left-Hand Navigation Enhancement**
+    *   Introduced an innovative dual-navigation approach for admin settings, providing both traditional top-nav tabs and a modern left-hand hierarchical navigation system.
+    *   **Key Features**: Conditional navigation that automatically detects layout preference, hierarchical structure with two-level navigation (tabs → sections), smart state management for active states and submenus.
+    *   **Comprehensive Organization**: All admin tabs now include organized sub-sections with proper section targeting for enhanced navigation.
+    *   **Benefits**: Matches conversation navigation patterns users already know, provides better organization for complex admin settings, enables bookmarkable deep links to specific sections.
+    *   (Ref: `admin_settings.html`, `_sidebar_nav.html`, `admin_sidebar_nav.js`)
+
+*   **Time-Based Logging Turnoff Feature**
+    *   Provides administrators with automatic turnoff capabilities for debug logging and file process logging to manage costs and security risks.
+    *   **Cost Management**: Prevents excessive logging costs by automatically disabling logging after specified time periods (minutes to weeks).
+    *   **Risk Mitigation**: Reduces security risks by ensuring debug logging doesn't remain enabled indefinitely.
+    *   **Configuration Options**: Supports time ranges from 1-120 minutes, 1-24 hours, 1-7 days, and 1-52 weeks for both debug logging and file processing logs.
+    *   **Background Monitoring**: Daemon thread monitors and enforces timer expiration automatically.
+    *   (Ref: `admin_settings.html`, `route_frontend_admin_settings.py`, `app.py`)
+
+*   **Comprehensive Table Support Enhancement**
+    *   Enhanced table rendering to support multiple input formats ensuring tables from AI agents or users are properly displayed as styled HTML tables.
+    *   **Format Support**: Unicode box-drawing tables (┌─┬─┐ style), markdown tables wrapped in code blocks, pipe-separated values (PSV) in code blocks, standard markdown tables.
+    *   **Processing Pipeline**: Implements preprocessing pipeline that detects and converts various table formats to standard markdown before parsing.
+    *   **Bootstrap Integration**: All generated tables automatically receive Bootstrap styling with striped rows and responsive design.
+    *   (Ref: `chat-messages.js`, table conversion functions, functional tests)
+
+*   **Public Workspace Management Enhancement**
+    *   Added "Go to Public Workspace" button to Public Workspace Management page for quick navigation from management to workspace usage.
+    *   **User Experience**: One-click navigation from management page to public workspace, automatically sets workspace as active for the user.
+    *   **Consistency**: Aligns with existing Group Workspace management functionality, provides consistent workflow between management and usage.
+    *   (Ref: `manage_public_workspace.html`, `route_frontend_public_workspaces.py`)
+
+*   **Multimedia Support Reorganization**
+    *   Reorganized Multimedia Support section from "Other" tab to "Search and Extract" tab with comprehensive Azure AI Video Indexer configuration guide.
+    *   **Enhanced Configuration**: Added detailed setup instructions modal with step-by-step account creation, API key acquisition guidelines, and troubleshooting section.
+    *   **Improved Organization**: Groups related search and extraction capabilities together, maintains all existing multimedia settings and functionality.
+    *   (Ref: `admin_settings.html`, `_video_indexer_info.html`)
 
 #### Bug Fixes
 
-##### Public Workspace Management Fixes
+*   **Admin Configuration Improvements**
+    *   Addressed user feedback about admin settings organization and implemented critical improvements to reduce confusion and provide better guidance.
+    *   **Duplicate Health Check Fix**: Consolidated health check configuration in General tab, removed duplicate from Other tab, added missing form field processing.
+    *   **Tab Organization**: Reorganized tabs into logical groups (Core Settings, AI Models Group, Content Processing Group, Security, User Features, System Administration).
+    *   **Workspace Dependency Validation**: Implemented real-time JavaScript validation to guide users when workspaces are enabled without required services (Azure AI Search, Document Intelligence, Embeddings).
+    *   (Ref: `admin_settings.html`, `admin_settings.js`, `route_frontend_admin_settings.py`, `route_external_health.py`)
+
+*   **Admin Settings Tab Preservation Fix**
+    *   Fixed issue where admin settings page would redirect to "General" tab after saving, rather than preserving the active tab.
+    *   **Root Cause**: Server-side redirects lose hash fragments, and tab activation only checked URL hash on page load without restoration mechanism.
+    *   **Solution**: Implemented client-side tab preservation using sessionStorage, enhanced with dual navigation interface support (traditional tabs and sidebar navigation).
+    *   **User Experience**: Users can now save settings and remain in their current tab, reducing frustration and improving workflow efficiency.
+    *   (Ref: `admin_settings.js`, tab restoration logic, session storage implementation)
+
+*   **Workspace Scope Prompts Fix**
+    *   Fixed workspace scope selector to affect both document filtering and prompt filtering consistently.
+    *   **Issue**: Workspace scope selection only affected documents but not prompts, creating inconsistent user experience.
+    *   **Solution**: Integrated prompt loading with workspace scope selector, implemented scope-aware filtering logic (All, Personal, Group, Public), added event listeners for scope changes.
+    *   **Impact**: Consistent behavior between document and prompt filtering, improved workflow efficiency for users working within specific workspace contexts.
+    *   (Ref: `chat-prompts.js`, `chat-global.js`, scope filtering implementation)
+
+*   **External Links New Window Fix**
+    *   Fixed web links in AI responses and user messages to open in new windows/tabs instead of replacing current chat session.
+    *   **Root Cause**: External links in markdown content didn't include `target="_blank"` attribute after DOMPurify sanitization.
+    *   **Solution**: Created `addTargetBlankToExternalLinks()` utility function that identifies external links and adds proper attributes including security measures.
+    *   **Security Enhancement**: Added `rel="noopener noreferrer"` for enhanced security, maintains DOMPurify sanitization.
+    *   (Ref: `chat-utils.js`, `chat-messages.js`, external link processing)
+
+*   **Video Indexer Debug Logging Enhancement**
+    *   Enhanced Video Indexer functionality with comprehensive debug logging to help diagnose API call failures and configuration issues.
+    *   **Comprehensive Logging**: Added detailed logging for authentication, upload process, processing polling, insights extraction, chunk processing, and video deletion.
+    *   **Troubleshooting Support**: Provides detailed error information, request/response data, and step-by-step processing details for customer support.
+    *   **Integration**: Uses existing `debug_print` function with `enable_debug_logging` setting for controlled debugging without performance impact.
+    *   (Ref: `functions_authentication.py`, `functions_documents.py`, Video Indexer workflow logging)
+
+### **(v0.229.014)**
+
+#### Bug Fixes
 
 *   **Public Workspace Management Permission Fix**
     *   Fixed incorrect permission checking for public workspace management operations when "Require Membership to Create Public Workspaces" setting was enabled.
@@ -24,8 +97,6 @@
     *   **Benefits**: Improved workspace identification, consistent with Group scope naming pattern, better navigation between workspace scopes.
     *   (Ref: `chat-documents.js`, scope label updates, dynamic workspace display)
 
-##### User Interface and Content Rendering Fixes
-
 *   **Unicode Table Rendering Fix**
     *   Fixed issue where AI-generated tables using Unicode box-drawing characters were not rendering as proper HTML tables in the chat interface.
     *   **Problem**: AI agents (particularly ESAM Agent) generated Unicode tables that appeared as plain text instead of formatted tables.

application/single_app/app.py

@@ -18,6 +18,10 @@
 from functions_settings import *
 from functions_appinsights import *
 
+import threading
+import time
+from datetime import datetime
+
 from route_frontend_authentication import *
 from route_frontend_profile import *
 from route_frontend_admin_settings import *
@@ -105,6 +109,69 @@ def before_first_request():
     print("Application initialized.")
     ensure_default_global_agent_exists()
 
+    # Background task to check for expired logging timers
+    def check_logging_timers():
+        """Background task that checks for expired logging timers and disables logging accordingly"""
+        while True:
+            try:
+                settings = get_settings()
+                current_time = datetime.now()
+                settings_changed = False
+
+                # Check debug logging timer
+                if (settings.get('enable_debug_logging', False) and 
+                    settings.get('debug_logging_timer_enabled', False) and 
+                    settings.get('debug_logging_turnoff_time')):
+
+                    turnoff_time = settings.get('debug_logging_turnoff_time')
+                    if isinstance(turnoff_time, str):
+                        try:
+                            turnoff_time = datetime.fromisoformat(turnoff_time)
+                        except:
+                            turnoff_time = None
+
+                    if turnoff_time and current_time >= turnoff_time:
+                        print(f"Debug logging timer expired at {turnoff_time}. Disabling debug logging.")
+                        settings['enable_debug_logging'] = False
+                        settings['debug_logging_timer_enabled'] = False
+                        settings['debug_logging_turnoff_time'] = None
+                        settings_changed = True
+
+                # Check file processing logs timer
+                if (settings.get('enable_file_processing_logs', False) and 
+                    settings.get('file_processing_logs_timer_enabled', False) and 
+                    settings.get('file_processing_logs_turnoff_time')):
+
+                    turnoff_time = settings.get('file_processing_logs_turnoff_time')
+                    if isinstance(turnoff_time, str):
+                        try:
+                            turnoff_time = datetime.fromisoformat(turnoff_time)
+                        except:
+                            turnoff_time = None
+
+                    if turnoff_time and current_time >= turnoff_time:
+                        print(f"File processing logs timer expired at {turnoff_time}. Disabling file processing logs.")
+                        settings['enable_file_processing_logs'] = False
+                        settings['file_processing_logs_timer_enabled'] = False
+                        settings['file_processing_logs_turnoff_time'] = None
+                        settings_changed = True
+
+                # Save settings if any changes were made
+                if settings_changed:
+                    update_settings(settings)
+                    print("Logging settings updated due to timer expiration.")
+
+            except Exception as e:
+                print(f"Error in logging timer check: {e}")
+
+            # Check every 60 seconds
+            time.sleep(60)
+
+    # Start the background timer check thread
+    timer_thread = threading.Thread(target=check_logging_timers, daemon=True)
+    timer_thread.start()
+    print("Logging timer background task started.")
+
 
     # Setup session handling
     if settings.get('enable_redis_cache'):
@@ -443,7 +510,7 @@ def list_semantic_kernel_plugins():
 
     if debug_mode:
         # Local development with HTTPS
-        app.run(host="0.0.0.0", port=5001, debug=True, ssl_context='adhoc')
+        app.run(host="0.0.0.0", port=5000, debug=True, ssl_context='adhoc')
     else:
         # Production
         port = int(os.environ.get("PORT", 5000))

application/single_app/config.py

@@ -88,7 +88,7 @@
 EXECUTOR_TYPE = 'thread'
 EXECUTOR_MAX_WORKERS = 30
 SESSION_TYPE = 'filesystem'
-VERSION = "0.229.019"
+VERSION = "0.229.058"
 
 SECRET_KEY = os.getenv('SECRET_KEY', 'dev-secret-key-change-in-production')
 

application/single_app/functions_authentication.py

@@ -250,6 +250,11 @@ def get_video_indexer_account_token(settings, video_id=None):
     2) POST to the ARM generateAccessToken endpoint
     3) Return the account-level accessToken
     """
+    from functions_debug import debug_print
+
+    debug_print(f"[VIDEO INDEXER AUTH] Starting token acquisition for video_id: {video_id}")
+    debug_print(f"[VIDEO INDEXER AUTH] Azure environment: {AZURE_ENVIRONMENT}")
+
     # 1) ARM token
     if AZURE_ENVIRONMENT == "usgovernment":
         arm_scope = "https://management.usgovcloudapi.net/.default"
@@ -258,16 +263,26 @@ def get_video_indexer_account_token(settings, video_id=None):
     else:
         arm_scope = "https://management.azure.com/.default"
 
-    credential = DefaultAzureCredential()
-    arm_token = credential.get_token(arm_scope).token
-    print("[VIDEO] ARM token acquired", flush=True)
+    debug_print(f"[VIDEO INDEXER AUTH] Using ARM scope: {arm_scope}")
+
+    try:
+        credential = DefaultAzureCredential()
+        debug_print(f"[VIDEO INDEXER AUTH] DefaultAzureCredential initialized successfully")
+        arm_token = credential.get_token(arm_scope).token
+        debug_print(f"[VIDEO INDEXER AUTH] ARM token acquired successfully (length: {len(arm_token) if arm_token else 0})")
+        print("[VIDEO] ARM token acquired", flush=True)
+    except Exception as e:
+        debug_print(f"[VIDEO INDEXER AUTH] ERROR acquiring ARM token: {str(e)}")
+        raise
 
     # 2) Call the generateAccessToken API
     rg       = settings["video_indexer_resource_group"]
     sub      = settings["video_indexer_subscription_id"]
     acct     = settings["video_indexer_account_name"]
     api_ver  = settings.get("video_indexer_arm_api_version", "2021-11-10-preview")
 
+    debug_print(f"[VIDEO INDEXER AUTH] Settings extracted - Subscription: {sub}, Resource Group: {rg}, Account: {acct}, API Version: {api_ver}")
+
     if AZURE_ENVIRONMENT == "usgovernment":
         url = (
         f"https://management.usgovcloudapi.net/subscriptions/{sub}"
@@ -290,22 +305,49 @@ def get_video_indexer_account_token(settings, video_id=None):
         f"/generateAccessToken?api-version={api_ver}"
         )
 
+    debug_print(f"[VIDEO INDEXER AUTH] ARM API URL: {url}")
+
     body = {
         "permissionType": "Contributor",
         "scope": "Account"
     }
     if video_id:
         body["videoId"] = video_id
 
-    resp = requests.post(
-        url,
-        json=body,
-        headers={"Authorization": f"Bearer {arm_token}"}
-    )
-    resp.raise_for_status()
-    ai = resp.json().get("accessToken")
-    print(f"[VIDEO] Account token acquired (len={len(ai)})", flush=True)
-    return ai
+    debug_print(f"[VIDEO INDEXER AUTH] Request body: {body}")
+
+    try:
+        resp = requests.post(
+            url,
+            json=body,
+            headers={"Authorization": f"Bearer {arm_token}"}
+        )
+        debug_print(f"[VIDEO INDEXER AUTH] ARM API response status: {resp.status_code}")
+
+        if resp.status_code != 200:
+            debug_print(f"[VIDEO INDEXER AUTH] ARM API response text: {resp.text}")
+
+        resp.raise_for_status()
+        response_data = resp.json()
+        debug_print(f"[VIDEO INDEXER AUTH] ARM API response keys: {list(response_data.keys())}")
+
+        ai = response_data.get("accessToken")
+        if not ai:
+            debug_print(f"[VIDEO INDEXER AUTH] ERROR: No accessToken in response: {response_data}")
+            raise ValueError("No accessToken found in ARM API response")
+
+        debug_print(f"[VIDEO INDEXER AUTH] Account token acquired successfully (length: {len(ai)})")
+        print(f"[VIDEO] Account token acquired (len={len(ai)})", flush=True)
+        return ai
+    except requests.exceptions.RequestException as e:
+        debug_print(f"[VIDEO INDEXER AUTH] ERROR in ARM API request: {str(e)}")
+        if hasattr(e, 'response') and e.response is not None:
+            debug_print(f"[VIDEO INDEXER AUTH] Error response status: {e.response.status_code}")
+            debug_print(f"[VIDEO INDEXER AUTH] Error response text: {e.response.text}")
+        raise
+    except Exception as e:
+        debug_print(f"[VIDEO INDEXER AUTH] Unexpected error: {str(e)}")
+        raise
 
 
 JWKS_CACHE = {}