Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

log-analytics workspace creation is failing at mcr.microsoft.com/azure-cli:latest but version 2.47.0 works #26297

Closed
v-vegujjula opened this issue Apr 28, 2023 · 12 comments · Fixed by #26287
Assignees
Labels
Auto-Assign Auto assign by bot Azure CLI Team The command of the issue is owned by Azure CLI team customer-reported Issues that are reported by GitHub users external to the Azure organization. Monitor az monitor question The issue doesn't require a change to the product in order to be resolved. Most issues start as that

Comments

@v-vegujjula
Copy link

az feedback auto-generates most of the information requested below, as of CLI version 2.0.62

Related command
az monitor log-analytics workspace create -g conformance-common -n analytics-vgr -l eastus

Describe the bug
While creating log analytics workspace at mcr.microsoft.com/azure-cli:latest image build container its failing. but its works at 2.47.0
ERROR
dummy-pod:/# az monitor log-analytics workspace create -g conformance-common -n analytics-vgr -l eastus --debug
cli.knack.cli: Command arguments: ['monitor', 'log-analytics', 'workspace', 'create', '-g', 'conformance-common', '-n', 'analytics-vgr', '-l', 'eastus', '--debug']
cli.knack.cli: init debug log:
Enable color in terminal.
cli.knack.cli: Event: Cli.PreExecute []
cli.knack.cli: Event: CommandParser.OnGlobalArgumentsCreate [<function CLILogging.on_global_arguments at 0x7f060b521360>, <function OutputProducer.on_global_arguments at 0x7f060b3fc040>, <function CLIQuery.on_global_arguments at 0x7f060b4252d0>]
cli.knack.cli: Event: CommandInvoker.OnPreCommandTableCreate []
cli.azure.cli.core: Modules found from index for 'monitor': ['azure.cli.command_modules.monitor']
cli.azure.cli.core: Loading command modules:
cli.azure.cli.core: Name Load Time Groups Commands
cli.azure.cli.core: monitor 0.092 32 98
cli.azure.cli.core: Total (1) 0.092 32 98
cli.azure.cli.core: These extensions are not installed and will be skipped: ['azext_ai_examples', 'azext_next']
cli.azure.cli.core: Loading extensions:
cli.azure.cli.core: Name Load Time Groups Commands Directory
cli.azure.cli.core: Total (0) 0.000 0 0
cli.azure.cli.core: Loaded 32 groups, 98 commands.
cli.azure.cli.core: Found a match in the command table.
cli.azure.cli.core: Raw command : monitor log-analytics workspace create
cli.azure.cli.core: Command table: monitor log-analytics workspace create
cli.knack.cli: Event: CommandInvoker.OnPreCommandTableTruncate [<function AzCliLogging.init_command_file_logging at 0x7f0609b59fc0>]
cli.azure.cli.core.azlogging: metadata file logging enabled - writing logs to '/root/.azure/commands/2023-04-28.17-28-12.monitor_log-analytics_workspace_create.40.log'.
az_command_data_logger: command args: monitor log-analytics workspace create -g {} -n {} -l {} --debug
cli.knack.cli: Event: CommandInvoker.OnPreArgumentLoad [<function register_global_subscription_argument..add_subscription_parameter at 0x7f0609b72b00>]
cli.knack.cli: Event: CommandInvoker.OnPostArgumentLoad []
cli.knack.cli: Event: CommandInvoker.OnPostCommandTableCreate [<function register_ids_argument..add_ids_arguments at 0x7f0609928a60>, <function register_cache_arguments..add_cache_arguments at 0x7f0609928b80>]
cli.knack.cli: Event: CommandInvoker.OnCommandTableLoaded []
cli.knack.cli: Event: CommandInvoker.OnPreParseArgs []
cli.knack.cli: Event: CommandInvoker.OnPostParseArgs [<function OutputProducer.handle_output_argument at 0x7f060b3fc0d0>, <function CLIQuery.handle_query_parameter at 0x7f060b425360>, <function register_ids_argument..parse_ids_arguments at 0x7f0609928af0>]
cli.azure.cli.core.auth.persistence: build_persistence: location='/root/.azure/service_principal_entries.json', encrypt=False
cli.azure.cli.core.auth.persistence: build_persistence: location='/root/.azure/msal_token_cache.json', encrypt=False
cli.azure.cli.core.auth.binary_cache: load: /root/.azure/msal_http_cache.bin
cli.azure.cli.core.decorators: <function BinaryCache._load at 0x7f0608831990> failed in No. 1 attempt
cli.azure.cli.core.decorators: Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/azure/cli/core/decorators.py", line 102, in _wrapped_func
return func(*args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/azure/cli/core/auth/binary_cache.py", line 35, in _load
return pickle.load(f)
File "/usr/local/lib/python3.10/site-packages/requests/models.py", line 28, in
from .auth import HTTPBasicAuth
File "/usr/local/lib/python3.10/site-packages/requests/init.py", line 135, in
from .models import Request, Response, PreparedRequest
ImportError: cannot import name 'Request' from partially initialized module 'requests.models' (most likely due to a circular import) (/usr/local/lib/python3.10/site-packages/requests/models.py)

cli.azure.cli.core.azclierror: Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/knack/cli.py", line 233, in invoke
cmd_result = self.invocation.execute(args)
File "/usr/local/lib/python3.10/site-packages/azure/cli/core/commands/init.py", line 663, in execute
raise ex
File "/usr/local/lib/python3.10/site-packages/azure/cli/core/commands/init.py", line 726, in _run_jobs_serially
results.append(self._run_job(expanded_arg, cmd_copy))
File "/usr/local/lib/python3.10/site-packages/azure/cli/core/commands/init.py", line 707, in _run_job
if _is_poller(result):
File "/usr/local/lib/python3.10/site-packages/azure/cli/core/commands/init.py", line 1156, in _is_poller
return isinstance(obj, poller_classes())
File "/usr/local/lib/python3.10/site-packages/azure/cli/core/util.py", line 686, in poller_classes
from msrestazure.azure_operation import AzureOperationPoller
File "/usr/local/lib/python3.10/site-packages/msrestazure/init.py", line 28, in
from .azure_configuration import AzureConfiguration
File "/usr/local/lib/python3.10/site-packages/msrestazure/azure_configuration.py", line 34, in
from msrest import Configuration
File "/usr/local/lib/python3.10/site-packages/msrest/init.py", line 28, in
from .configuration import Configuration
File "/usr/local/lib/python3.10/site-packages/msrest/configuration.py", line 37, in
from .pipeline import Pipeline
File "/usr/local/lib/python3.10/site-packages/msrest/pipeline/init.py", line 52, in
from requests.structures import CaseInsensitiveDict
File "/usr/local/lib/python3.10/site-packages/requests/init.py", line 135, in
from .models import Request, Response, PreparedRequest
ImportError: cannot import name 'Request' from partially initialized module 'requests.models' (most likely due to a circular import) (/usr/local/lib/python3.10/site-packages/requests/models.py)

cli.azure.cli.core.azclierror: The command failed with an unexpected error. Here is the traceback:
az_command_data_logger: The command failed with an unexpected error. Here is the traceback:
cli.azure.cli.core.azclierror: cannot import name 'Request' from partially initialized module 'requests.models' (most likely due to a circular import) (/usr/local/lib/python3.10/site-packages/requests/models.py)
Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/knack/cli.py", line 233, in invoke
cmd_result = self.invocation.execute(args)
File "/usr/local/lib/python3.10/site-packages/azure/cli/core/commands/init.py", line 663, in execute
raise ex
File "/usr/local/lib/python3.10/site-packages/azure/cli/core/commands/init.py", line 726, in _run_jobs_serially
results.append(self._run_job(expanded_arg, cmd_copy))
File "/usr/local/lib/python3.10/site-packages/azure/cli/core/commands/init.py", line 707, in _run_job
if _is_poller(result):
File "/usr/local/lib/python3.10/site-packages/azure/cli/core/commands/init.py", line 1156, in _is_poller
return isinstance(obj, poller_classes())
File "/usr/local/lib/python3.10/site-packages/azure/cli/core/util.py", line 686, in poller_classes
from msrestazure.azure_operation import AzureOperationPoller
File "/usr/local/lib/python3.10/site-packages/msrestazure/init.py", line 28, in
from .azure_configuration import AzureConfiguration
File "/usr/local/lib/python3.10/site-packages/msrestazure/azure_configuration.py", line 34, in
from msrest import Configuration
File "/usr/local/lib/python3.10/site-packages/msrest/init.py", line 28, in
from .configuration import Configuration
File "/usr/local/lib/python3.10/site-packages/msrest/configuration.py", line 37, in
from .pipeline import Pipeline
File "/usr/local/lib/python3.10/site-packages/msrest/pipeline/init.py", line 52, in
from requests.structures import CaseInsensitiveDict
File "/usr/local/lib/python3.10/site-packages/requests/init.py", line 135, in
from .models import Request, Response, PreparedRequest
ImportError: cannot import name 'Request' from partially initialized module 'requests.models' (most likely due to a circular import) (/usr/local/lib/python3.10/site-packages/requests/models.py)
az_command_data_logger: cannot import name 'Request' from partially initialized module 'requests.models' (most likely due to a circular import) (/usr/local/lib/python3.10/site-packages/requests/models.py)
Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/knack/cli.py", line 233, in invoke
cmd_result = self.invocation.execute(args)
File "/usr/local/lib/python3.10/site-packages/azure/cli/core/commands/init.py", line 663, in execute
raise ex
File "/usr/local/lib/python3.10/site-packages/azure/cli/core/commands/init.py", line 726, in _run_jobs_serially
results.append(self._run_job(expanded_arg, cmd_copy))
File "/usr/local/lib/python3.10/site-packages/azure/cli/core/commands/init.py", line 707, in _run_job
if _is_poller(result):
File "/usr/local/lib/python3.10/site-packages/azure/cli/core/commands/init.py", line 1156, in _is_poller
return isinstance(obj, poller_classes())
File "/usr/local/lib/python3.10/site-packages/azure/cli/core/util.py", line 686, in poller_classes
from msrestazure.azure_operation import AzureOperationPoller
File "/usr/local/lib/python3.10/site-packages/msrestazure/init.py", line 28, in
from .azure_configuration import AzureConfiguration
File "/usr/local/lib/python3.10/site-packages/msrestazure/azure_configuration.py", line 34, in
from msrest import Configuration
File "/usr/local/lib/python3.10/site-packages/msrest/init.py", line 28, in
from .configuration import Configuration
File "/usr/local/lib/python3.10/site-packages/msrest/configuration.py", line 37, in
from .pipeline import Pipeline
File "/usr/local/lib/python3.10/site-packages/msrest/pipeline/init.py", line 52, in
from requests.structures import CaseInsensitiveDict
File "/usr/local/lib/python3.10/site-packages/requests/init.py", line 135, in
from .models import Request, Response, PreparedRequest
ImportError: cannot import name 'Request' from partially initialized module 'requests.models' (most likely due to a circular import) (/usr/local/lib/python3.10/site-packages/requests/models.py)
To check existing issues, please visit: https://github.com/Azure/azure-cli/issues
To open a new issue, please run az feedback
cli.knack.cli: Event: Cli.PostExecute [<function AzCliLogging.deinit_cmd_metadata_logging at 0x7f0609b5a200>]
az_command_data_logger: exit code: 1
cli.main: Command ran in 0.631 seconds (init: 0.293, invoke: 0.338)
telemetry.main: Begin splitting cli events and extra events, total events: 1
telemetry.client: Accumulated 0 events. Flush the clients.
telemetry.main: Finish splitting cli events and extra events, cli events: 1
telemetry.save: Save telemetry record of length 7258 in cache
telemetry.check: Negative: The /root/.azure/telemetry.txt was modified at 2023-04-28 17:26:27.832464, which in less than 600.000000 s

To Reproduce
create a docker image with below docker file format and run the image. Inside container execute az login --service-principal -u (with credentials) and execute az monitor log-analytics workspace create -g conformance-common -n analytics-vgr -l eastus

FROM mcr.microsoft.com/azure-cli:2.47.0
RUN pip install --trusted-host pypi.org --trusted-host files.pythonhosted.org pytest pytest-xdist filelock junit_xml kubernetes azure.identity
RUN pip install azure.mgmt.resource
RUN apk add gnupg ca-certificates curl wget jq

Expected behavior
dummy-pod:/# az monitor log-analytics workspace create -g conformance-common -n analytics-vgr -l eastus
{
"createdDate": "2023-04-28T19:22:13.9499021Z",
"customerId": "16f16f1b-a11d-495d-8e8d-c6932c1cbb14",
"features": {
"enableLogAccessUsingOnlyResourcePermissions": true
},
"id": "/subscriptions/3959ec86-5353-4b0c-b5d7-3877122861a0/resourceGroups/conformance-common/providers/Microsoft.OperationalInsights/workspaces/analytics-vgr",
"location": "eastus",
"modifiedDate": "2023-04-28T19:22:13.9499021Z",
"name": "analytics-vgr",
"provisioningState": "Succeeded",
"publicNetworkAccessForIngestion": "Enabled",
"publicNetworkAccessForQuery": "Enabled",
"resourceGroup": "conformance-common",
"retentionInDays": 30,
"sku": {
"lastSkuUpdate": "2023-04-28T19:22:13.9499021Z",
"name": "PerGB2018"
},
"tags": {
"resourceTag": "addedByPolicy"
},
"type": "Microsoft.OperationalInsights/workspaces",
"workspaceCapping": {
"dailyQuotaGb": -1.0,
"dataIngestionStatus": "RespectQuota",
"quotaNextResetTime": "2023-04-29T17:00:00Z"
}
}

Environment summary
azure-cli                         2.48.1
core                              2.48.1
telemetry                          1.0.8
Extensions:
arcdata                           1.4.13
connectedk8s                      1.3.16
customlocation                     0.1.3
k8s-configuration                  1.7.0
k8s-extension                      1.4.0

Dependencies:
msal                              1.20.0
azure-mgmt-resource               22.0.0

Python location '/usr/local/bin/python'
Extensions directory '/root/.azure/cliextensions'

Python (Linux) 3.10.11 (main, Apr  5 2023, 23:58:40) [GCC 12.2.1 20220924]

Legal docs and information: aka.ms/AzureCliLegal
Your CLI is up-to-date.

Additional context

@ghost ghost added customer-reported Issues that are reported by GitHub users external to the Azure organization. Auto-Assign Auto assign by bot Monitor az monitor labels Apr 28, 2023
@ghost ghost assigned AllyW Apr 28, 2023
@yonzhan
Copy link
Collaborator

yonzhan commented Apr 28, 2023

Thank you for opening this issue, we will look into it.

@ghost ghost added this to the Backlog milestone Apr 28, 2023
@ghost ghost added Azure CLI Team The command of the issue is owned by Azure CLI team question The issue doesn't require a change to the product in order to be resolved. Most issues start as that Account az login/account labels Apr 28, 2023
@ghost ghost assigned jiasli Apr 28, 2023
@ghost ghost added the Installation label Apr 28, 2023
@yonzhan yonzhan removed Account az login/account Installation labels Apr 28, 2023
@v-vegujjula
Copy link
Author

@yonzhan
did you get a chance to look in this. Let me know if any further information is required.

@marlowp
Copy link

marlowp commented May 2, 2023

This error is also occurring when attempting to delete a Log Analytics Workspace with the latest CLI version.

Error Details

ERROR: The command failed with an unexpected error. Here is the traceback: ERROR: cannot import name 'Request' from partially initialized module 'requests.models' (most likely due to a circular import) (/opt/az/lib/python3.10/site-packages/requests/models.py) Traceback (most recent call last):   File "/opt/az/lib/python3.10/site-packages/knack/cli.py", line 233, in invoke     cmd_result = self.invocation.execute(args)   File "/opt/az/lib/python3.10/site-packages/azure/cli/core/commands/__init__.py", line 663, in execute     raise ex   File "/opt/az/lib/python3.10/site-packages/azure/cli/core/commands/__init__.py", line 726, in _run_jobs_serially     results.append(self._run_job(expanded_arg, cmd_copy))   File "/opt/az/lib/python3.10/site-packages/azure/cli/core/commands/__init__.py", line 707, in _run_job     if _is_poller(result):   File "/opt/az/lib/python3.10/site-packages/azure/cli/core/commands/__init__.py", line 1156, in _is_poller     return isinstance(obj, poller_classes())   File "/opt/az/lib/python3.10/site-packages/azure/cli/core/util.py", line 686, in poller_classes     from msrestazure.azure_operation import AzureOperationPoller   File "/opt/az/lib/python3.10/site-packages/msrestazure/__init__.py", line 28, in <module>     from .azure_configuration import AzureConfiguration   File "/opt/az/lib/python3.10/site-packages/msrestazure/azure_configuration.py", line 34, in <module>     from msrest import Configuration   File "/opt/az/lib/python3.10/site-packages/msrest/__init__.py", line 28, in <module>     from .configuration import Configuration   File "/opt/az/lib/python3.10/site-packages/msrest/configuration.py", line 37, in <module>     from .pipeline import Pipeline   File "/opt/az/lib/python3.10/site-packages/msrest/pipeline/__init__.py", line 52, in <module>     from requests.structures import CaseInsensitiveDict   File "/opt/az/lib/python3.10/site-packages/requests/__init__.py", line 135, in <module>     from .models import Request, Response, PreparedRequest ImportError: cannot import name 'Request' from partially initialized module 'requests.models' (most likely due to a circular import) (/opt/az/lib/python3.10/site-packages/requests/models.py) To check existing issues, please visit: https://github.com/Azure/azure-cli/issues To open a new issue, please run `az feedback`

@marlowp
Copy link

marlowp commented May 2, 2023

Reverting to CLI version 2.47.0 or 2.46.0 has not resolved this issue for us. We're running Microsoft hosted agents using ubuntu-20.04 (also tried ubuntu-22.04) and we're reverting the CLI version via:

pip install -Iv azure-cli==2.47.0 --extra-index-url https://azurecliprod.blob.core.windows.net/edge

At the moment our only workaround is to completely remove the usage of a Log Analytics workspace from our CICD pipelines which is not ideal.

@AllyW
Copy link
Member

AllyW commented May 2, 2023

Hi @v-vegujjula @marlowp , we are dealing with this import failure. And for short-time solution, you can try to run rm commandIndex.json under azure cached files' folder, (which is usually ~/.azure or the AZURE_CONFIG_DIR user set), before every az monitor log-analytics workspace create\delete cmd runs.

@v-vegujjula
Copy link
Author

@AllyW
Yes I have tested above suggested way ie working fine. But as part of my automation workflow it need changes at multiple places. Do we have any ETA on this permanent fix ?

@marlowp
Copy link

marlowp commented May 3, 2023

@AllyW thanks, that solution does work.

For those using the AzureCLI pipeline task keep in mind that the useGlobalConfig input will determine whether that specific CLI task uses global config in the ~/.azure directory or create its own. If you're using powershell the following will delete the file:

Remove-Item "$Env:AZURE_CONFIG_DIR/commandIndex.json" -Force -Verbose

@v-vegujjula
Copy link
Author

@AllyW Yes I have tested above suggested way ie working fine. But as part of my automation workflow it need changes at multiple places. Do we have any ETA on this permanent fix ?

Any update on permanent fix ?

@kairu-ms
Copy link
Contributor

kairu-ms commented May 4, 2023

This issue will be resolved in the release of May 23.

@kairu-ms
Copy link
Contributor

kairu-ms commented May 4, 2023

@AllyW Yes I have tested above suggested way ie working fine. But as part of my automation workflow it need changes at multiple places. Do we have any ETA on this permanent fix ?

Hi @v-vegujjula, you can set this environment in your script AZURE_CORE_USE_COMMAND_INDEX=False. It's the same as deleting the commandIndex.json file.

@v-vegujjula
Copy link
Author

This issue will be resolved in the release of May 23.

@kairu-ms
thanks for sharing the update. If possible can you please share the release date.

@yonzhan
Copy link
Collaborator

yonzhan commented May 4, 2023

May 23

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Auto-Assign Auto assign by bot Azure CLI Team The command of the issue is owned by Azure CLI team customer-reported Issues that are reported by GitHub users external to the Azure organization. Monitor az monitor question The issue doesn't require a change to the product in order to be resolved. Most issues start as that
Projects
None yet
6 participants