CloudFormation: How to handle errors in UserData-CodePudding

As part of my CloudFormation template, I leverage UserData to run a series of commands that install my application. I want to know if there's a way to respond to an exit error when UserData is run so that the EC2 instance terminated and doesn't get added to my Auto Scaling Group.

I looked into cfn-init and cfn-signal which I thought could provide this functionality, but looking through examples I only see these used in conjunction with metadata configSets.

This is what UserData under my AWS::AutoScaling::LaunchConfiguration looks like:

"WebServerLaunchConfig": {
    "Type": "AWS::AutoScaling::LaunchConfiguration",
    "Metadata": ...,
    "Properties": {
        ...
        "UserData": {"Fn::Base64": {
            "Fn::Join": [
            "\n",
            [
                "#!/bin/bash -xe",
                "\n",
                "cd /var/www"
                "git clone myrepo html",
                "cd /var/www/html",
                "/usr/bin/aws s3 cp s3://my-bucket/env.development .env",
                ... // several more commands, etc.
            ]
        ]}
        }
    }
}

At a high level, my CF structure looks like:

Load Balancer -> Auto Scaling Group for EC2 instances (desired 1, min 1, max 3)

I have dynamic scaling policies in my ASG to scale up and down depending on load. So let's say the scale up policy is triggered. A new EC2 instance will be created and put into the pool. If for some reason there is a failure in the UserData portion of my stack, the EC2 instance will still be created successfully. I would like some sort of alarm triggered if UserData has a bad exit code so that the instance can be terminated and I can be notified.

Is this possible?

CodePudding user response：

This is something that should be configured at the UserData level. Think of the UserData as simply a shell script which runs on the machine. You have full control over what happens while the script is run. Few options come to mind to achieve the behavior you are looking for:

Setup the script so that on error it sends an alarm to CloudWatch
Setup the script so that on error it uses SNS/SES to send you an email
Setup the script so that on error the instance shuts itself down automatically

(or indeed you can combine the above).

As to how the shell script should be configured, I suggest the use of trap, so that you can easily run whatever logic you want if the script ever errors.

An example:

#!/bin/bash -xe

notify() {
  echo "UserData was unsuccessful!"
  ... 
  # use this function to implement the notification/shutdown behavior
}

trap 'notify' ERR

cd /var/www
git clone myrepo html
...

CodePudding user response：

I would like some sort of alarm triggered if UserData has a bad exit code so that the instance can be terminated and I can be notified.

This is something you have to program yourself in your UserData. AWS does not have auto-of-the-box means to check for errors in your UserData.

CodePudding user response：

Yes I think you are trying to restart Nginx in your userData hence you can modify your asg( target group) health check to check the response from the created ec2 is coming from your started nginx , if it matches its considered healthy and if its not running its considered unhealthy and replaced