Combination of azure-queue and cron scaler not working as expected.

[WolfgangHBaeck]

I have the following triggers for my scaler:

triggers:

• type: azure-queue
  metadata:
  queueName: classification-queue
  queueLength: '40'
  activationQueueLength: '0'
  connectionFromEnv: AZURE_STORAGE_QUEUE_CONN_STRING
  accountName: {{ .Values.accntqueue_name }}
  cloud: AzurePublicCloud
• type: cron
  metadata:
  desiredReplicas: "1"
  start: 0 11 * * 1-5
  end: 0 3 * * *
  timezone: UTC

My intention is to allow the queue scaler to scale between 0 and "maxReplicaCount" any time except during the start through end period during the week days. It's okay to scale to 0 based on queue length any other time.
The reason for the limitation during the working week is that the scaling up from 0 is more time consuming and I like to avoid it except for the weekend.
I am suspecting that I have something wrong here. It never scales to 0 during the weekend even if the queue is empty.