Skip to content

Latest commit

 

History

History
230 lines (178 loc) · 4.99 KB

New-AzHDInsightStreamingMapReduceJobDefinition.md

File metadata and controls

230 lines (178 loc) · 4.99 KB
Raw
external help file Module Name ms.assetid online version schema
Microsoft.Azure.PowerShell.Cmdlets.HDInsight.dll-Help.xml
Az.HDInsight
17CB76E7-2F91-4EFE-9DA3-F083F02235E1
2.0.0

New-AzHDInsightStreamingMapReduceJobDefinition

SYNOPSIS

Creates a Streaming MapReduce job object.

SYNTAX

New-AzHDInsightStreamingMapReduceJobDefinition [-Arguments <String[]>] [-File <String>] [-Files <String[]>]
 [-StatusFolder <String>] [-CommandEnvironment <Hashtable>] [-Defines <Hashtable>] -InputPath <String>
 [-Mapper <String>] [-OutputPath <String>] [-Reducer <String>] [-DefaultProfile <IAzureContextContainer>]
 [<CommonParameters>]

DESCRIPTION

The New-AzHDInsightStreamingMapReduceJobDefinition cmdlet defines a Streaming MapReduce job object for use with an Azure HDInsight cluster.

EXAMPLES

Example 1: Create a Streaming MapReduce job definition

# Cluster info
$clusterName = "your-hadoop-001"
$clusterCreds = Get-Credential

# Streaming MapReduce job details
$statusFolder = "tempStatusFolder/"
$query = "SHOW TABLES"

New-AzHDInsightStreamingMapReduceJobDefinition -StatusFolder $statusFolder -File $query `
        | Start-AzHDInsightJob `
            -ClusterName $clusterName `
            -ClusterCredential $clusterCreds

This command creates a Streaming MapReduce job definition.

PARAMETERS

-Arguments

Specifies an array of arguments for the job. The arguments are passed as command-line arguments to each task.

Type: System.String[]
Parameter Sets: (All)
Aliases:

Required: False
Position: Named
Default value: None
Accept pipeline input: False
Accept wildcard characters: False

-CommandEnvironment

Specifies an array of command-line environment variables to set when a job runs on worker nodes.

Type: System.Collections.Hashtable
Parameter Sets: (All)
Aliases:

Required: False
Position: Named
Default value: None
Accept pipeline input: False
Accept wildcard characters: False

-DefaultProfile

The credentials, account, tenant, and subscription used for communication with azure

Type: Microsoft.Azure.Commands.Common.Authentication.Abstractions.Core.IAzureContextContainer
Parameter Sets: (All)
Aliases: AzContext, AzureRmContext, AzureCredential

Required: False
Position: Named
Default value: None
Accept pipeline input: False
Accept wildcard characters: False

-Defines

Specifies Hadoop configuration values to set for when the job runs.

Type: System.Collections.Hashtable
Parameter Sets: (All)
Aliases:

Required: False
Position: Named
Default value: None
Accept pipeline input: False
Accept wildcard characters: False

-File

Specifies the path to a file that contains a query to run. You can use this parameter instead of the Query parameter.

Type: System.String
Parameter Sets: (All)
Aliases:

Required: False
Position: Named
Default value: None
Accept pipeline input: False
Accept wildcard characters: False

-Files

Specifies a collection of files that are associated with a Hive job.

Type: System.String[]
Parameter Sets: (All)
Aliases:

Required: False
Position: Named
Default value: None
Accept pipeline input: False
Accept wildcard characters: False

-InputPath

Specifies the path to the input files.

Type: System.String
Parameter Sets: (All)
Aliases:

Required: True
Position: Named
Default value: None
Accept pipeline input: False
Accept wildcard characters: False

-Mapper

Specifies a Mapper file name.

Type: System.String
Parameter Sets: (All)
Aliases:

Required: False
Position: Named
Default value: None
Accept pipeline input: False
Accept wildcard characters: False

-OutputPath

Specifies the path for the job output.

Type: System.String
Parameter Sets: (All)
Aliases:

Required: False
Position: Named
Default value: None
Accept pipeline input: False
Accept wildcard characters: False

-Reducer

Specifies a Reducer file name.

Type: System.String
Parameter Sets: (All)
Aliases:

Required: False
Position: Named
Default value: None
Accept pipeline input: False
Accept wildcard characters: False

-StatusFolder

Specifies the location of the folder that contains standard outputs and error outputs for a job.

Type: System.String
Parameter Sets: (All)
Aliases:

Required: False
Position: Named
Default value: None
Accept pipeline input: False
Accept wildcard characters: False

CommonParameters

This cmdlet supports the common parameters: -Debug, -ErrorAction, -ErrorVariable, -InformationAction, -InformationVariable, -OutVariable, -OutBuffer, -PipelineVariable, -Verbose, -WarningAction, and -WarningVariable. For more information, see about_CommonParameters.

INPUTS

None

OUTPUTS

Microsoft.Azure.Commands.HDInsight.Models.AzureHDInsightStreamingMapReduceJobDefinition

NOTES

RELATED LINKS

Start-AzHDInsightJob