Terraform does (not) need your code to provision a lambda function

TLDR;

Terraform expects a deployment package before it can provision a lambda function

The deployment package can actually be any non-empty zip file

Once the lambda has been created you should deploy actual code using a CI/CD pipeline

Consider using a dedicated serverless framework, but define clear boundaries

Use a Terraform module for provisioning lambda functions to reduce effort

‍

Terraform is a great infrastructure-as-code tool which we love at Ensono Digital, but effectively implementing the aws_lambda_function resource in the real world can be a little bit challenging. I am going to explain an approach for provisioning lambda functions without needing a pre-built deployment package.

‍

Terraform and lambda – chicken or the egg?

The lambda function resource is a bit special, as it requires a suitable deployment package containing your function code to exist before it can go ahead and create the function. This contrasts to typical Terraform resources, and infrastructure as code in general, where you can stand-up resources in advance. The necessary deployment package can be in the form of a local file or an s3 object.

(This is requirement comes from the underlying AWS CreateFunction API action.)

‍

Local file vs s3 object

If you have a function which is using an interpreted language, and the code is never going to change, then storing a pre-packaged deployment package alongside your Terraform and using the local “filename” option is probably going to suffice. In which case, move along, nothing to see here.

However, functions which using a compiled language, which get built and packaged in a CI pipeline are going to require a more creative solution.

‍

What Terraform wants you to do

What Terraform wants you to do feels something like this:

‍

‍

Great, except providing actual code to Terraform just feelswrong for many reasons:

Last time I checked, Terraform was an Infrastructure-as-codetool. Infrastructure! Since when did you have to provide application codeto create infrastructure?

There is a circular dependency between the function codeCI/CD pipeline and Terraform

You need to ensure Terraform always references the correct versionof the deployment package for each environment, this might not always bethe latest

How do you ensure Terraform doesn’t subsequently “update”the function code outside of your CI/CD process?

Surely there must be a better way….

‍

What I want to do

I want to create the infrastructure (specifically, an empty shell of a lambda function, with it’s corresponding IAM role, triggers, log configuration, permissions, etc) first, so that a separate CI/CD pipeline can then build, package, and deploy the function code. Like this

‍

‍

Implementation

My first plan was to simply trick Terraform into provisioning my lambda function by giving it an empty zip file, so I whipped out the trusty archive_file provider and pointed it to an empty dir and fired away:

Not so fast! Looks like someone has thought of this already, and the AWS API swiftly rejects my file with the following message:

OK then, so let’s just add some content into the zip file:

‍

‍

It worked!

Now obviously any invocations of the lambda function will fail at this point, but it’s now super easy to deploy your actual code to the function from a CI/CD pipeline completely independently from Terraform, for example:

aws lambda update-function-code –function-name $function_name –zip-file fileb://$source_path –publish

Next steps

Now that you are ready to create the lambda function, You probably need a whole host of other resources to complement it into an actual working

IAM (aws_iam_role / aws_iam_role_policy)

event source mapping for SQS, DynamoDB, Kinesis (aws_lambda_event_source_mapping)

bucket notification for s3 (aws_s3_bucket_notification)

cloudwatch trigger for scheduling (aws_cloudwatch_event_rule, aws_cloudwatch_event_target)

permissions for invocations from other resources (aws_lambda_permission)

‍

You can encapsulate this collection of resources into a re-usable Terraform module to reduce the effort required. Or you could use one which already exists.

Alternatives

There are frameworks available which are designed specifically for implementing serverless architectures. These do a lot of the heavy lifting by assisting with creation of supporting resources and configuration such as IAM roles, triggers, permissions, etc. In addition to provisioning resources, they can also handle packaging and deployment of the function code, so if you have a significant amount of lambda functions, should really check them out:

• Serverless Framework
• AWS Serverless Application Model (SAM)
• Apex

There is overlap between Terraform and the “Serverless” / “SAM frameworks and as these can all provision supporting resources as well as the lambda function itself. Consider how the rest of your infrastructure is going to be managed, as if you end up using Terraform and the Serverless framework, this will cause some fragmentation in tooling. For example, what if you also have a significant container infrastructure provisioned using Terraform, but some containers share resources (s3, sqs, etc) with lambda functions, where should the shared resources get provisioned? Terraform or the Serverless framework?

‍

‍

‍

‍

‍