Skip to content
Permalink
Branch: master
Find file Copy path
Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
265 lines (203 sloc) 6.49 KB

external help file: Microsoft.Azure.PowerShell.Cmdlets.HDInsight.dll-Help.xml Module Name: Az.HDInsight ms.assetid: 3C6DCC81-82F7-4044-AFBC-4EE1BCC306F2 online version: https://docs.microsoft.com/en-us/powershell/module/az.hdinsight/invoke-azhdinsighthivejob schema: 2.0.0

Invoke-AzHDInsightHiveJob

SYNOPSIS

Submits a Hive query to an HDInsight cluster and retrieves query results in one operation.

SYNTAX

Invoke-AzHDInsightHiveJob [-Arguments <String[]>] [-Files <String[]>] [-StatusFolder <String>]
 [-Defines <Hashtable>] [-File <String>] [-JobName <String>] [-Query <String>] [-RunAsFileJob]
 [-DefaultContainer <String>] [-DefaultStorageAccountName <String>] [-DefaultStorageAccountKey <String>]
 [-DefaultProfile <IAzureContextContainer>] [<CommonParameters>]

DESCRIPTION

The Invoke-AzHDInsightHiveJob cmdlet submits a Hive query to an Azure HDInsight cluster and retrieves query results in one operation. Use the Use-AzHDInsightCluster cmdlet before calling Invoke-AzHDInsightHiveJob to specify which cluster will be used for the query.

EXAMPLES

Example 1: Submit a Hive query to an Azure HDInsight cluster

PS C:\># Primary storage account info
PS C:\> $storageAccountResourceGroupName = "Group"
PS C:\> $storageAccountName = "yourstorageacct001"
PS C:\> $storageAccountKey = (Get-AzStorageAccountKey -ResourceGroupName $storageAccountResourceGroupName -Name $storageAccountName)[0].value


PS C:\> $storageContainer = "container001"

# Cluster info
PS C:\> $clusterName = "your-hadoop-001"
PS C:\> $clusterCreds = Get-Credential

# Hive job details
PS C:\> $statusFolder = "tempStatusFolder/"
PS C:\> $query = "SHOW TABLES"

PS C:\> Use-AzHDInsightCluster `
            -ClusterCredential $clusterCreds `
            -ClusterName $clusterName

PS C:\> Invoke-AzHDInsightHiveJob -StatusFolder $statusFolder `
            -Query $query `
            -DefaultContainer $storageAccountContainer `
            -DefaultStorageAccountName "$storageAccountName.blob.core.windows.net" `
            -DefaultStorageAccountKey $storageAccountKey

This command submits the query SHOW TABLES to the cluster named your-hadoop-001.

PARAMETERS

-Arguments

Specifies an array of arguments for the job. The arguments are passed as command-line arguments to each task.

Type: System.String[]
Parameter Sets: (All)
Aliases:

Required: False
Position: Named
Default value: None
Accept pipeline input: False
Accept wildcard characters: False

-DefaultContainer

Specifies the name of the default container in the default Azure Storage account that an HDInsight cluster uses.

Type: System.String
Parameter Sets: (All)
Aliases:

Required: False
Position: Named
Default value: None
Accept pipeline input: False
Accept wildcard characters: False

-DefaultProfile

The credentials, account, tenant, and subscription used for communication with azure

Type: Microsoft.Azure.Commands.Common.Authentication.Abstractions.Core.IAzureContextContainer
Parameter Sets: (All)
Aliases: AzContext, AzureRmContext, AzureCredential

Required: False
Position: Named
Default value: None
Accept pipeline input: False
Accept wildcard characters: False

-DefaultStorageAccountKey

Specifies the account key for the default storage account that the HDInsight cluster uses.

Type: System.String
Parameter Sets: (All)
Aliases:

Required: False
Position: Named
Default value: None
Accept pipeline input: False
Accept wildcard characters: False

-DefaultStorageAccountName

Specifies the name of the default storage account that the HDInsight cluster uses.

Type: System.String
Parameter Sets: (All)
Aliases:

Required: False
Position: Named
Default value: None
Accept pipeline input: False
Accept wildcard characters: False

-Defines

Specifies Hadoop configuration values to set when a job runs.

Type: System.Collections.Hashtable
Parameter Sets: (All)
Aliases:

Required: False
Position: Named
Default value: None
Accept pipeline input: False
Accept wildcard characters: False

-File

Specifies the path to a file in Azure Storage that contains the query to run. You can use this parameter instead of the Query parameter.

Type: System.String
Parameter Sets: (All)
Aliases:

Required: False
Position: Named
Default value: None
Accept pipeline input: False
Accept wildcard characters: False

-Files

Specifies a collection of files that are required for a Hive job.

Type: System.String[]
Parameter Sets: (All)
Aliases:

Required: False
Position: Named
Default value: None
Accept pipeline input: False
Accept wildcard characters: False

-JobName

Specifies the name of a Hive job. If you do not specify this parameter, this cmdlet uses the default value: "Hive: <first 100 characters of Query>".

Type: System.String
Parameter Sets: (All)
Aliases:

Required: False
Position: Named
Default value: None
Accept pipeline input: False
Accept wildcard characters: False

-Query

Specifies the Hive query.

Type: System.String
Parameter Sets: (All)
Aliases:

Required: False
Position: Named
Default value: None
Accept pipeline input: False
Accept wildcard characters: False

-RunAsFileJob

Indicates that this cmdlet creates a file in the default Azure storage account in which to store a query. This cmdlet submits the job that references this file as a script to run. You can use this functionality to handle special characters such as percent sign (%) that would fail on a job submission through Templeton, because Templeton interprets a query with a percent sign as a URL parameter.

Type: System.Management.Automation.SwitchParameter
Parameter Sets: (All)
Aliases:

Required: False
Position: Named
Default value: None
Accept pipeline input: False
Accept wildcard characters: False

-StatusFolder

Specifies the location of the folder that contains standard outputs and error outputs for a job.

Type: System.String
Parameter Sets: (All)
Aliases:

Required: False
Position: Named
Default value: None
Accept pipeline input: False
Accept wildcard characters: False

CommonParameters

This cmdlet supports the common parameters: -Debug, -ErrorAction, -ErrorVariable, -InformationAction, -InformationVariable, -OutVariable, -OutBuffer, -PipelineVariable, -Verbose, -WarningAction, and -WarningVariable. For more information, see about_CommonParameters (http://go.microsoft.com/fwlink/?LinkID=113216).

INPUTS

None

OUTPUTS

System.String

NOTES

RELATED LINKS

Use-AzHDInsightCluster

You can’t perform that action at this time.