Creates an Autopilot job.
Find the best-performing model after you run an Autopilot job by calling .
For information about how to use Autopilot, see Automate Model Development with Amazon SageMaker Autopilot .
See also: AWS API Documentation
create-auto-ml-job
--auto-ml-job-name <value>
--input-data-config <value>
--output-data-config <value>
[--problem-type <value>]
[--auto-ml-job-objective <value>]
[--auto-ml-job-config <value>]
--role-arn <value>
[--generate-candidate-definitions-only | --no-generate-candidate-definitions-only]
[--tags <value>]
[--model-deploy-config <value>]
[--cli-input-json | --cli-input-yaml]
[--generate-cli-skeleton <value>]
[--debug]
[--endpoint-url <value>]
[--no-verify-ssl]
[--no-paginate]
[--output <value>]
[--query <value>]
[--profile <value>]
[--region <value>]
[--version <value>]
[--color <value>]
[--no-sign-request]
[--ca-bundle <value>]
[--cli-read-timeout <value>]
[--cli-connect-timeout <value>]
[--cli-binary-format <value>]
[--no-cli-pager]
[--cli-auto-prompt]
[--no-cli-auto-prompt]
--auto-ml-job-name
(string)
Identifies an Autopilot job. The name must be unique to your account and is case-insensitive.
--input-data-config
(list)
An array of channel objects that describes the input data and its location. Each channel is a named input source. Similar to
InputDataConfig
supported by . Format(s) supported: CSV, Parquet. A minimum of 500 rows is required for the training dataset. There is not a minimum number of rows required for the validation dataset.(structure)
A channel is a named input source that training algorithms can consume. The validation dataset size is limited to less than 2 GB. The training dataset size must be less than 100 GB. For more information, see .
Note
A validation dataset must contain the same headers as the training dataset.
DataSource -> (structure)
The data source for an AutoML channel.
S3DataSource -> (structure)
The Amazon S3 location of the input data.
S3DataType -> (string)
The data type.
A ManifestFile should have the format shown below:
[ {"prefix": "s3://DOC-EXAMPLE-BUCKET/DOC-EXAMPLE-FOLDER/DOC-EXAMPLE-PREFIX/"},
"DOC-EXAMPLE-RELATIVE-PATH/DOC-EXAMPLE-FOLDER/DATA-1",
"DOC-EXAMPLE-RELATIVE-PATH/DOC-EXAMPLE-FOLDER/DATA-2",
... "DOC-EXAMPLE-RELATIVE-PATH/DOC-EXAMPLE-FOLDER/DATA-N" ]
An S3Prefix should have the following format:
s3://DOC-EXAMPLE-BUCKET/DOC-EXAMPLE-FOLDER-OR-FILE
S3Uri -> (string)
The URL to the Amazon S3 data source.
CompressionType -> (string)
You can use
Gzip
orNone
. The default value isNone
.TargetAttributeName -> (string)
The name of the target variable in supervised learning, usually represented by ‘y’.
ContentType -> (string)
The content type of the data from the input source. You can use
text/csv;header=present
orx-application/vnd.amazon+parquet
. The default value istext/csv;header=present
.ChannelType -> (string)
The channel type (optional) is an
enum
string. The default value istraining
. Channels for training and validation must share the sameContentType
andTargetAttributeName
. For information on specifying training and validation channel types, see `How to specify training and validation datasets
https://docs.aws.amazon.com/sagemaker/latest/dg/autopilot-datasets-problem-types.html#autopilot-data-sources-training-or-validation`__ .Shorthand Syntax:
DataSource={S3DataSource={S3DataType=string,S3Uri=string}},CompressionType=string,TargetAttributeName=string,ContentType=string,ChannelType=string ...JSON Syntax:
[ { "DataSource": { "S3DataSource": { "S3DataType": "ManifestFile"|"S3Prefix", "S3Uri": "string" } }, "CompressionType": "None"|"Gzip", "TargetAttributeName": "string", "ContentType": "string", "ChannelType": "training"|"validation" } ... ]
--output-data-config
(structure)Provides information about encryption and the Amazon S3 output path needed to store artifacts from an AutoML job. Format(s) supported: CSV.
KmsKeyId -> (string)
The Key Management Service (KMS) encryption key ID.
S3OutputPath -> (string)
The Amazon S3 output path. Must be 128 characters or less.
Shorthand Syntax:
KmsKeyId=string,S3OutputPath=stringJSON Syntax:
{ "KmsKeyId": "string", "S3OutputPath": "string" }
--problem-type
(string)Defines the type of supervised learning available for the candidates. For more information, see Amazon SageMaker Autopilot problem types and algorithm support .
Possible values:
BinaryClassification
MulticlassClassification
Regression
--auto-ml-job-objective
(structure)Defines the objective metric used to measure the predictive quality of an AutoML job. You provide an AutoMLJobObjective$MetricName and Autopilot infers whether to minimize or maximize it.
MetricName -> (string)
The name of the objective metric used to measure the predictive quality of a machine learning system. This metric is optimized during training to provide the best estimate for model parameter values from data.
Here are the options:
Accuracy
The ratio of the number of correctly classified items to the total number of (correctly and incorrectly) classified items. It is used for both binary and multiclass classification. Accuracy measures how close the predicted class values are to the actual values. Values for accuracy metrics vary between zero (0) and one (1). A value of 1 indicates perfect accuracy, and 0 indicates perfect inaccuracy.
AUC
The area under the curve (AUC) metric is used to compare and evaluate binary classification by algorithms that return probabilities, such as logistic regression. To map the probabilities into classifications, these are compared against a threshold value.
The relevant curve is the receiver operating characteristic curve (ROC curve). The ROC curve plots the true positive rate (TPR) of predictions (or recall) against the false positive rate (FPR) as a function of the threshold value, above which a prediction is considered positive. Increasing the threshold results in fewer false positives, but more false negatives.
AUC is the area under this ROC curve. Therefore, AUC provides an aggregated measure of the model performance across all possible classification thresholds. AUC scores vary between 0 and 1. A score of 1 indicates perfect accuracy, and a score of one half (0.5) indicates that the prediction is not better than a random classifier.
BalancedAccuracy
BalancedAccuracy
is a metric that measures the ratio of accurate predictions to all predictions. This ratio is calculated after normalizing true positives (TP) and true negatives (TN) by the total number of positive (P) and negative (N) values. It is used in both binary and multiclass classification and is defined as follows: 0.5*((TP/P)+(TN/N)), with values ranging from 0 to 1.BalancedAccuracy
gives a better measure of accuracy when the number of positives or negatives differ greatly from each other in an imbalanced dataset. For example, when only 1% of email is spam.F1
The
F1
score is the harmonic mean of the precision and recall, defined as follows: F1 = 2 * (precision * recall) / (precision + recall). It is used for binary classification into classes traditionally referred to as positive and negative. Predictions are said to be true when they match their actual (correct) class, and false when they do not.Precision is the ratio of the true positive predictions to all positive predictions, and it includes the false positives in a dataset. Precision measures the quality of the prediction when it predicts the positive class.
Recall (or sensitivity) is the ratio of the true positive predictions to all actual positive instances. Recall measures how completely a model predicts the actual class members in a dataset.
F1 scores vary between 0 and 1. A score of 1 indicates the best possible performance, and 0 indicates the worst.
F1macro
The
F1macro
score applies F1 scoring to multiclass classification problems. It does this by calculating the precision and recall, and then taking their harmonic mean to calculate the F1 score for each class. Lastly, the F1macro averages the individual scores to obtain theF1macro
score.F1macro
scores vary between 0 and 1. A score of 1 indicates the best possible performance, and 0 indicates the worst.MAE
The mean absolute error (MAE) is a measure of how different the predicted and actual values are, when they’re averaged over all values. MAE is commonly used in regression analysis to understand model prediction error. If there is linear regression, MAE represents the average distance from a predicted line to the actual value. MAE is defined as the sum of absolute errors divided by the number of observations. Values range from 0 to infinity, with smaller numbers indicating a better model fit to the data.
MSE
The mean squared error (MSE) is the average of the squared differences between the predicted and actual values. It is used for regression. MSE values are always positive. The better a model is at predicting the actual values, the smaller the MSE value is
Precision
Precision measures how well an algorithm predicts the true positives (TP) out of all of the positives that it identifies. It is defined as follows: Precision = TP/(TP+FP), with values ranging from zero (0) to one (1), and is used in binary classification. Precision is an important metric when the cost of a false positive is high. For example, the cost of a false positive is very high if an airplane safety system is falsely deemed safe to fly. A false positive (FP) reflects a positive prediction that is actually negative in the data.
PrecisionMacro
The precision macro computes precision for multiclass classification problems. It does this by calculating precision for each class and averaging scores to obtain precision for several classes.
PrecisionMacro
scores range from zero (0) to one (1). Higher scores reflect the model’s ability to predict true positives (TP) out of all of the positives that it identifies, averaged across multiple classes.R2
R2, also known as the coefficient of determination, is used in regression to quantify how much a model can explain the variance of a dependent variable. Values range from one (1) to negative one (-1). Higher numbers indicate a higher fraction of explained variability.
R2
values close to zero (0) indicate that very little of the dependent variable can be explained by the model. Negative values indicate a poor fit and that the model is outperformed by a constant function. For linear regression, this is a horizontal line.Recall
Recall measures how well an algorithm correctly predicts all of the true positives (TP) in a dataset. A true positive is a positive prediction that is also an actual positive value in the data. Recall is defined as follows: Recall = TP/(TP+FN), with values ranging from 0 to 1. Higher scores reflect a better ability of the model to predict true positives (TP) in the data, and is used in binary classification.
Recall is important when testing for cancer because it’s used to find all of the true positives. A false positive (FP) reflects a positive prediction that is actually negative in the data. It is often insufficient to measure only recall, because predicting every output as a true positive will yield a perfect recall score.
RecallMacro
The RecallMacro computes recall for multiclass classification problems by calculating recall for each class and averaging scores to obtain recall for several classes. RecallMacro scores range from 0 to 1. Higher scores reflect the model’s ability to predict true positives (TP) in a dataset. Whereas, a true positive reflects a positive prediction that is also an actual positive value in the data. It is often insufficient to measure only recall, because predicting every output as a true positive will yield a perfect recall score.
RMSE
Root mean squared error (RMSE) measures the square root of the squared difference between predicted and actual values, and it’s averaged over all values. It is used in regression analysis to understand model prediction error. It’s an important metric to indicate the presence of large model errors and outliers. Values range from zero (0) to infinity, with smaller numbers indicating a better model fit to the data. RMSE is dependent on scale, and should not be used to compare datasets of different sizes.
If you do not specify a metric explicitly, the default behavior is to automatically use:
MSE
: for regression.
F1
: for binary classification
Accuracy
: for multiclass classification.Shorthand Syntax:
MetricName=stringJSON Syntax:
{ "MetricName": "Accuracy"|"MSE"|"F1"|"F1macro"|"AUC"|"RMSE"|"MAE"|"R2"|"BalancedAccuracy"|"Precision"|"PrecisionMacro"|"Recall"|"RecallMacro" }
--auto-ml-job-config
(structure)A collection of settings used to configure an AutoML job.
CompletionCriteria -> (structure)
How long an AutoML job is allowed to run, or how many candidates a job is allowed to generate.
MaxCandidates -> (integer)
The maximum number of times a training job is allowed to run.
MaxRuntimePerTrainingJobInSeconds -> (integer)
The maximum time, in seconds, that each training job executed inside hyperparameter tuning is allowed to run as part of a hyperparameter tuning job. For more information, see the used by the action.
MaxAutoMLJobRuntimeInSeconds -> (integer)
The maximum runtime, in seconds, an AutoML job has to complete.
If an AutoML job exceeds the maximum runtime, the job is stopped automatically and its processing is ended gracefully. The AutoML job identifies the best model whose training was completed and marks it as the best-performing model. Any unfinished steps of the job, such as automatic one-click Autopilot model deployment, will not be completed.
SecurityConfig -> (structure)
The security configuration for traffic encryption or Amazon VPC settings.
VolumeKmsKeyId -> (string)
The key used to encrypt stored data.
EnableInterContainerTrafficEncryption -> (boolean)
Whether to use traffic encryption between the container layers.
VpcConfig -> (structure)
The VPC configuration.
SecurityGroupIds -> (list)
The VPC security group IDs, in the form sg-xxxxxxxx. Specify the security groups for the VPC that is specified in the
Subnets
field.(string)
Subnets -> (list)
The ID of the subnets in the VPC to which you want to connect your training job or model. For information about the availability of specific instance types, see Supported Instance Types and Availability Zones .
(string)
DataSplitConfig -> (structure)
The configuration for splitting the input training dataset.
Type: AutoMLDataSplitConfig
ValidationFraction -> (float)
The validation fraction (optional) is a float that specifies the portion of the training dataset to be used for validation. The default value is 0.2, and values must be greater than 0 and less than 1. We recommend setting this value to be less than 0.5.
CandidateGenerationConfig -> (structure)
The configuration for generating a candidate for an AutoML job (optional).
FeatureSpecificationS3Uri -> (string)
A URL to the Amazon S3 data source containing selected features from the input data source to run an Autopilot job. You can input
FeatureAttributeNames
(optional) in JSON format as shown below:
{ "FeatureAttributeNames":["col1", "col2", ...] }
.You can also specify the data type of the feature (optional) in the format shown below:
{ "FeatureDataTypes":{"col1":"numeric", "col2":"categorical" ... } }
Note
These column keys may not include the target column.
In ensembling mode, Autopilot will only support the following data types:
numeric
,categorical
,text
anddatetime
. In HPO mode, Autopilot can supportnumeric
,categorical
,text
,datetime
andsequence
.If only
FeatureDataTypes
is provided, the column keys (col1
,col2
,..) should be a subset of the column names in the input data.If both
FeatureDataTypes
andFeatureAttributeNames
are provided, then the column keys should be a subset of the column names provided inFeatureAttributeNames
.The key name
FeatureAttributeNames
is fixed. The values listed in["col1", "col2", ...]
is case sensitive and should be a list of strings containing unique values that are a subset of the column names in the input data. The list of columns provided must not include the target column.Mode -> (string)
The method that Autopilot uses to train the data. You can either specify the mode manually or let Autopilot choose for you based on the dataset size by selecting
AUTO
. InAUTO
mode, Autopilot choosesENSEMBLING
for datasets smaller than 100 MB, andHYPERPARAMETER_TUNING
for larger ones.The
ENSEMBLING
mode uses a multi-stack ensemble model to predict classification and regression tasks directly from your dataset. This machine learning mode combines several base models to produce an optimal predictive model. It then uses a stacking ensemble method to combine predictions from contributing members. A multi-stack ensemble model can provide better performance over a single model by combining the predictive capabilities of multiple models. See Autopilot algorithm support for a list of algorithms supported byENSEMBLING
mode.The
HYPERPARAMETER_TUNING
(HPO) mode uses the best hyperparameters to train the best version of a model. HPO will automatically select an algorithm for the type of problem you want to solve. Then HPO finds the best hyperparameters according to your objective metric. See Autopilot algorithm support for a list of algorithms supported byHYPERPARAMETER_TUNING
mode.JSON Syntax:
{ "CompletionCriteria": { "MaxCandidates": integer, "MaxRuntimePerTrainingJobInSeconds": integer, "MaxAutoMLJobRuntimeInSeconds": integer }, "SecurityConfig": { "VolumeKmsKeyId": "string", "EnableInterContainerTrafficEncryption": true|false, "VpcConfig": { "SecurityGroupIds": ["string", ...], "Subnets": ["string", ...] } }, "DataSplitConfig": { "ValidationFraction": float }, "CandidateGenerationConfig": { "FeatureSpecificationS3Uri": "string" }, "Mode": "AUTO"|"ENSEMBLING"|"HYPERPARAMETER_TUNING" }
--role-arn
(string)The ARN of the role that is used to access the data.
--generate-candidate-definitions-only
|--no-generate-candidate-definitions-only
(boolean)Generates possible candidates without training the models. A candidate is a combination of data preprocessors, algorithms, and algorithm parameter settings.
--tags
(list)Each tag consists of a key and an optional value. Tag keys must be unique per resource.
(structure)
A tag object that consists of a key and an optional value, used to manage metadata for SageMaker Amazon Web Services resources.
You can add tags to notebook instances, training jobs, hyperparameter tuning jobs, batch transform jobs, models, labeling jobs, work teams, endpoint configurations, and endpoints. For more information on adding tags to SageMaker resources, see AddTags .
For more information on adding metadata to your Amazon Web Services resources with tagging, see Tagging Amazon Web Services resources . For advice on best practices for managing Amazon Web Services resources with tagging, see Tagging Best Practices: Implement an Effective Amazon Web Services Resource Tagging Strategy .
Key -> (string)
The tag key. Tag keys must be unique per resource.
Value -> (string)
The tag value.
Shorthand Syntax:
Key=string,Value=string ...JSON Syntax:
[ { "Key": "string", "Value": "string" } ... ]
--model-deploy-config
(structure)Specifies how to generate the endpoint name for an automatic one-click Autopilot model deployment.
AutoGenerateEndpointName -> (boolean)
Set to
True
to automatically generate an endpoint name for a one-click Autopilot model deployment; set toFalse
otherwise. The default value isFalse
.Note
If you set
AutoGenerateEndpointName
toTrue
, do not specify theEndpointName
; otherwise a 400 error is thrown.EndpointName -> (string)
Specifies the endpoint name to use for a one-click Autopilot model deployment if the endpoint name is not generated automatically.
Note
Specify the
EndpointName
if and only if you setAutoGenerateEndpointName
toFalse
; otherwise a 400 error is thrown.Shorthand Syntax:
AutoGenerateEndpointName=boolean,EndpointName=stringJSON Syntax:
{ "AutoGenerateEndpointName": true|false, "EndpointName": "string" }
--cli-input-json
|--cli-input-yaml
(string) Reads arguments from the JSON string provided. The JSON string follows the format provided by--generate-cli-skeleton
. If other arguments are provided on the command line, those values will override the JSON-provided values. It is not possible to pass arbitrary binary values using a JSON-provided value as the string will be taken literally. This may not be specified along with--cli-input-yaml
.
--generate-cli-skeleton
(string) Prints a JSON skeleton to standard output without sending an API request. If provided with no value or the valueinput
, prints a sample input JSON that can be used as an argument for--cli-input-json
. Similarly, if providedyaml-input
it will print a sample input YAML that can be used with--cli-input-yaml
. If provided with the valueoutput
, it validates the command inputs and returns a sample output JSON for that command. The generated JSON skeleton is not stable between versions of the AWS CLI and there are no backwards compatibility guarantees in the JSON skeleton generated.Global Options¶
--debug
(boolean)Turn on debug logging.
--endpoint-url
(string)Override command’s default URL with the given URL.
--no-verify-ssl
(boolean)By default, the AWS CLI uses SSL when communicating with AWS services. For each SSL connection, the AWS CLI will verify SSL certificates. This option overrides the default behavior of verifying SSL certificates.
--no-paginate
(boolean)Disable automatic pagination.
--output
(string)The formatting style for command output.
json
text
table
yaml
yaml-stream
--query
(string)A JMESPath query to use in filtering the response data.
--profile
(string)Use a specific profile from your credential file.
--region
(string)The region to use. Overrides config/env settings.
--version
(string)Display the version of this tool.
--color
(string)Turn on/off color output.
on
off
auto
--no-sign-request
(boolean)Do not sign requests. Credentials will not be loaded if this argument is provided.
--ca-bundle
(string)The CA certificate bundle to use when verifying SSL certificates. Overrides config/env settings.
--cli-read-timeout
(int)The maximum socket read time in seconds. If the value is set to 0, the socket read will be blocking and not timeout. The default value is 60 seconds.
--cli-connect-timeout
(int)The maximum socket connect time in seconds. If the value is set to 0, the socket connect will be blocking and not timeout. The default value is 60 seconds.
--cli-binary-format
(string)The formatting style to be used for binary blobs. The default format is base64. The base64 format expects binary blobs to be provided as a base64 encoded string. The raw-in-base64-out format preserves compatibility with AWS CLI V1 behavior and binary values must be passed literally. When providing contents from a file that map to a binary blob
fileb://
will always be treated as binary and use the file contents directly regardless of thecli-binary-format
setting. When usingfile://
the file contents will need to properly formatted for the configuredcli-binary-format
.
base64
raw-in-base64-out
--no-cli-pager
(boolean)Disable cli pager for output.
--cli-auto-prompt
(boolean)Automatically prompt for CLI input parameters.
--no-cli-auto-prompt
(boolean)Disable automatically prompt for CLI input parameters.
Output¶
AutoMLJobArn -> (string)
The unique ARN assigned to the AutoML job when it is created.