How to use Lambda as a glue?

2023-02-09 7 min read 3sky

Welcome

Let’s keep this article short. The topic is rather popular, but not very well documented. It’s based on quite a popular issue, which is about untagged EC2. Sometimes developers launch virtual machines, without tags. Why it’s so important? Because without tags, you have no idea about the machine’s purpose. You have no owner, now, project, no cost center, etc. I know that some of these things can be checked with CloudTrail, however, it’s still about keeping the environment clean. After kindly asking, reminders, etc, I decided to go with a slightly different kind of solution. The brutal one.

Intro

What is the brutal solution then? It’s simple if a new machine will be spawned without mandatory tags, an instance will be immediately terminated. Probably you have seen a lot of GUI-based tutorials for it. Most of them are great, if you want to implement a similar solution then, please go there. I would like to focus on governance, an as code solution. Also, I want to meet SAM.

What the SAM is? In simple words, it’s an AWS alternative to the well-known Serverless Framework. It provides app skeletons, dummy EventBridge events, unit and integration tests, and basic code in many popular languages. Also, it allows users to define all needed resources, with CloudFormation templates, and an easy deployment tool.

However please remember that this is not a magic solution and in many cases, a few adjustments will be needed. And that’s what I will include in this article. SAM intro with a little tweaking.

Tools used in this episode

SAM CLI
Python
CloudFormation(scripts)

Implementation

First, we need to install SAM. Fortunately it’s well documented via Amazon, and can be shortened to:

# linux
wget https://github.com/aws/aws-sam-cli/releases/latest/download/aws-sam-cli-linux-x86_64.zip -O aws-sam-cli-linux-x86_64.zip
unzip aws-sam-cli-linux-x86_64.zip -d sam-installation
sudo ./sam-installation/install

# mac
brew tap aws/tap
brew install aws-sam-cli

Init the project

Great, now we’re able to spin the project with one simple command. Nevertheless, I strongly recommend running through regular creators and exploring multiple options. For my instance stopper project, the bootstrap command looks like that:

sam init --name instance_stopper \
        --runtime python3.9 \
        --dependency-manager pip \
        --app-template eventBridge-hello-world \
        --architecture arm64

I think that the most important flag here is runtime and app-template. Python runtime is good, for my case, as It’s easy to understand and fast to implement. Maybe someday I will rewrite it into Go, however today I want simplicity. App-template from another hand download pre-define application template. After all, the script will produce a falling directory tree.

tree instance_stopper/
instance_stopper/
├── README.md
├── __init__.py
├── conftest.py
├── events
│   └── event.json
├── hello_world_function
│   ├── __init__.py
│   ├── hello_world
│   │   ├── __init__.py
│   │   └── app.py
│   ├── model
│   │   ├── __init__.py
│   │   └── aws
│   │       ├── __init__.py
│   │       └── ec2
│   │           ├── __init__.py
│   │           ├── aws_event.py
│   │           ├── ec2_instance_state_change_notification.py
│   │           └── marshaller.py
│   └── requirements.txt
├── template.yaml
└── tests
    ├── __init__.py
    ├── integration
    │   ├── __init__.py
    │   └── test_ec2_event.py
    ├── requirements.txt
    └── unit
        ├── __init__.py
        └── test_handler.py

10 directories, 21 files

The trip

Ok, let’s run some fast directory structure overview.

events - contains an example EventBridge event, useful for testing
hello_world_function\hello_world - main app directory
hello_world_function\model - event model, for data deserialization
template.yaml - CloudFormation template
tests\integration - integration test which requires a real environment
tests\unit - unit tests, PyTest for local validation

Python code

The first thing you need to take a look at is general logic. What do we want to archive? In my case, the flow is simple. If someone spins the instance, without the tag Owner, an instance will be immediately terminated. How to do it? With the usage of the boto3 library again!

At the beginning please remember about python venv and fetching requirements for testing.

python3 -m venv deploy
source deploy/bin/activate
pip install -r  tests/requirements.txt

Then let’s write some code:

# cat instance_stopper_function/instance_stopper/app.py
import boto3

from model.aws.ec2 import Marshaller
from model.aws.ec2 import AWSEvent
from model.aws.ec2 import EC2InstanceStateChangeNotification


def lambda_handler(event, context):

    # Deserialize event into strongly typed object - yea its possible
    awsEvent, ec2StateChangeNotification = deserialize_event(event)

    # use boto3 as client
    ec2 = boto3.resource('ec2')
    instance = ec2.Instance(ec2StateChangeNotification.instance_id)
    tags: dict = instance.tags

    # Execute business logic
    if search_owner(tags) is None:
        print("Kill it: " + ec2StateChangeNotification.instance_id)
        # Termiante instance with Owner tag
        instance.terminate()

        # Make updates to event payload
        awsEvent.detail_type = "Lambda function terminated the machine " + ec2StateChangeNotification.instance_id + " " + awsEvent.detail_type

    # Return event for further processing
    return Marshaller.marshall(awsEvent)


def search_owner(tags):

    if (next((x for x in tags if x["Key"] == "Owner"), None)) is None:
        return None
    else:
        return "Owner exist"


def deserialize_event(event):

    awsEvent: AWSEvent = Marshaller.unmarshall(event, AWSEvent)
    ec2StateChangeNotification: EC2InstanceStateChangeNotification = awsEvent.detail

    print("Region is " + awsEvent.region)
    print("Instance " + ec2StateChangeNotification.instance_id + " transitioned to " + ec2StateChangeNotification.state)

    return awsEvent, ec2StateChangeNotification

Probably you can see, that code it’s slightly different than the original. Also, I modified unit tests, because of that.

# cat tests/unit/test_handler.py
import pytest

from instance_stopper import app
from model.aws.ec2.ec2_instance_state_change_notification import EC2InstanceStateChangeNotification


@pytest.fixture()
def eventBridgeec2InstanceEvent():
    """ Generates EventBridge EC2 Instance Notification Event"""

    return {
            "version": "0",
            "id": "7bf73129-1428-4cd3-a780-95db273d1602",
            "detail-type": "EC2 Instance State-change Notification",
            "source": "aws.ec2",
            "account": "123456789012",
            "time": "2015-11-11T21:29:54Z",
            "region": "us-east-1",
            "resources": [
              "arn:aws:ec2:us-east-1:123456789012:instance/i-abcd1111"
            ],
            "detail": {
              "instance-id": "i-abcd1111",
              "state": "pending"
            }
    }


def test_lambda_handler(eventBridgeec2InstanceEvent):

    awsEvent, details = app.deserialize_event(eventBridgeec2InstanceEvent)

    assert awsEvent.region == "us-east-1"
    assert details.instance_id == "i-abcd1111"
    assert details.state == "pending"


def test_tag_filter():

    tags: list = [
            {'Key': 'Owner', 'Value': 'Kuba'},
            {'Key': 'Project', 'Value': 'Infra'},
            {'Key': 'Managedby', 'Value': 'SAM'},
            ]
    owner = app.search_owner(tags)
    assert owner.startswith("Owner")


def test_wrong_tags_filter():

    tags: list = [
            {'Key': 'Project', 'Value': 'Infra'},
            {'Key': 'Managedby', 'Value': 'SAM'},
            ]

    owner = app.search_owner(tags)
    assert owner is None

After writing all this code, let’s just test it.

pytest .
============================================================================= test session starts =============================================================================
platform darwin -- Python 3.10.9, pytest-7.2.1, pluggy-1.0.0
rootdir: /Users/kuba/Code/sam/instance_stopper
plugins: mock-3.10.0
collected 3 items

tests/unit/test_handler.py ...                                                                                                                                          [100%]

============================================================================== 3 passed in 0.08s ==============================================================================

In the end, I decided to skip integration tests. The task is too simple, to spin the whole stack, for testing.

AWS environment

Now, let’s take a look at template.yaml. Things which was changed:

Tagging - as it will be funny to skip it!
State of EventBridge, as I wanted to track the transition from pending to running state.

      Events:
        ChangeStateEvent:
          Type: CloudWatchEvent
          Properties:
            Pattern:
              source:
                - aws.ec2
              detail-type:
                - EC2 Instance State-change Notification
              detail:
                state:
                  - running

Then IAM Role. Initially, SAM provides a basic lambda execution role. Our case needs more complex scope.

  InstanceStopperRole:
    Type: 'AWS::IAM::Role'
    Properties:
      Tags:
        - Key: Owner
          Value: kuba
        - Key: Usecase
          Value: Infra
        - Key: ManagedBy
          Value: SAM
      AssumeRolePolicyDocument:
        Version: "2012-10-17"
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - lambda.amazonaws.com
            Action:
              - 'sts:AssumeRole'
      Path: /
      Policies:
        - PolicyName: StopTheInstance
          PolicyDocument:
            Version: "2012-10-17"
            Statement:
              - Effect: Allow
                Action:
                  - "logs:CreateLogGroup"
                  - "logs:CreateLogStream"
                  - "logs:PutLogEvents"
                Resource: "arn:aws:logs:*:*:*"
              - Effect: Allow
                Action:
                  - "ec2:DescribeInstances"
                  - "ec2:TerminateInstances"
                Resource: "*"

And that’s it! All changes. Now we can go perform the deployment.

Deployment

That part is awesome. We can deploy all these objects with managed SAM stack, with just one command. At first time, --guided mode could be helpful.

sam deploy --guided

After that, we will receive samconfig.toml file with the whole deployment configuration.

In general, we’re done. Lambda Function with all needed resources was deployed, and the whole config is stored as a code, so it’s waiting for basic CI/CD, which will be provided (maybe) in the future.

After all, we can just list our lambdas:

aws lambda list-functions --query='Functions[]'

Or logs:

sam logs --stack-name instance_stopper

Summary

In a few above steps and less than 20 minutes, we built and deployed Lambda Function with prerequisites, waiting for EventBridge events. Maybe it wasn’t fast and simple, but the whole stack was delivered. Easy to move, reuse, or modify. Also, it’s a CloudFormation stack, not GUI based function.

What next? As I said, CI/CD is missing, so we can investigate the best tool for small projects. Also, we can add some notifications, which will be sent to the developer, as well as to the admin. All communication can also be put in some database as a record of illegal usage.

Ah, and the project, which I built can be found here.