The Job Designer application enables you to create and submit Hadoop Map/Reduce jobs to the Hadoop cluster. You can include variables with your jobs to enable you and other users to enter values for the variables when they run your job. The Job Designer supports streaming, MapReduce, and Java jobs.
![]() | Note
|
Job Designer is one of the applications that is installed as part of Hue. For more information about installing Hue, see Hue Installation. For information about installing Oozie, see Oozie installation.
To start Job Designer, click the Job Designer tab in the navigation bar at the top of the Hue web page. The Job Designs page opens in the browser.
The Job Designer sample jobs can help you learn how to use Job Designer.
To install the Job Designer samples, click Install Samples in the Job Designs window and then click Yes. The sample jobs are displayed in the Job Designs window. Job Designer removes the Install Samples button after the samples are installed so you can only install the samples once.
In the Job Designer, a job design specifies several meta-level properties of a MapReduce job, including the job design name, description, the MapReduce executable scripts or classes, and any parameters for those scripts or classes. You can create three types of job designs: a MapReduce Job, a streaming job and a Java job design.
A Hadoop JAR consists of MapReduce functions written in Java.
To create a MapReduce job design:
Setting | Description |
---|---|
Name | The Name identifies the MapReduce job and it's collection of parameters. |
Description | Specify a description of the MapReduce job. The description is displayed in the dialog box that appears if you specify variables for the job. |
Jar path | Specify the name of the JAR file, including the path. |
Job Properties | Click Add Property, to set a property value: » For Property name, enter a configuration variable name. This field provides auto-completion, so you can type the first few characters of a property name and then select the one you want from a list. » For Value, enter the value you want to use for the variable. |
Files | Click Add File, then enter a file name or browse for a file to upload. |
Archives | Click Add Archive, then enter an archive name or browse for a |
Hadoop streaming jobs enable you to create MapReduce functions in any non-Java language that reads standard Unix input and writes standard Unix output. For more information about Hadoop streaming jobs, see needs updated link to info.
To create a streaming job design:
![]() | Note: You can use variables of the form $variable_name for the Mapper, and Reducer settings described in the following table. When the streaming job is run, a dialog box will appear to enable you or users to specify the values of the variables. |
Setting | Description |
---|---|
Name | The Name identifies the streaming job design including the associated properties and parameters. |
Description | Specify a description of the streaming job design. The description is displayed in the dialog box that appears if you specify variables for the job. |
Mapper | Specify the path to the mapper script or class. If the mapper file is not on the machines on the cluster, use the Files option to pack it as a part of job submission. Equivalent to the Hadoop -mapper option. |
Reducer | Specify the path to the reducer script or class. If the reducer file is not on the machines on the cluster, use the Files option to pack it as a part of job submission. Equivalent to the Hadoop -reducer option. |
Job Properties | Click Add Property, to set a property value: » For Property name, enter a configuration variable name. » For Value, enter the value you want to use for the variable. |
Files | Click Add File, then enter a file name what files? |
Archives | Click Add Archive, then enter an archive name what is this? |
A Hadoop JAR consists of MapReduce functions written in Java.
how is this different from a MapReduce job?
To create a Java job design:
![]() | Note: You can use variables of the form $variable_name for the Args setting described in the following table. When the JAR job is run, a dialog box will appear to enable you or users to specify the values of the variables. |
Setting | Description |
---|---|
Name | The Name identifies the Java job and it's collection of parameters. |
Description | Specify a description of the java job. The description is displayed in the dialog box that appears if you specify variables for the job. |
Jar path | Specify the name of the JAR file, including the path. |
Main class | Specify the (name of?) the Main class to invoke the program. |
Args | Specify the arguments you want to pass to the running JAR job. |
Java opts | Specify any Java options you want to specify as part of the job. |
Job Properties | Click Add Property, to set a property value: » For Property name, enter a configuration variable name. » For Value, enter the value you want to use for the variable. |
Files | Click Add File, then enter a file name what files? |
Archives | Click Add Archive, then enter an archive name what is this? |
To submit a job to a cluster:
If you want to edit and use a job but you don't own it, you can make a copy of it and then edit and use the copied job.
To copy a job design:
To edit a job design:
To delete a job design:
You can filter the Job Designs by owner, by job name, or both.
To filter the Job Designs list:
To display the Job Submission History