How to build and deploy NestJS Lambda functions with the AWS SAM CLI
This article builds on a great tutorial on AWS SAM and Typescript by Andrey Novikov and Sergey Alexandrovich of Evil Martians.
Their tutorial centers around using a custom Makefile when building AWS serverless giving the benefit or far greater customization and control.
Specifically, node_modules can be separated into a Lambda layer reducing function bundle size.
I thought this was a great start but I wanted to add NestJS into the mix and have Nest be the entry point for my lambda functions.
There are some tutorials focused on connecting NestJS with a lambda entry point but my personal preference (so far) is from Peter Morlion’s post where NestFactory.createApplicationContext is used in place of traditional bootstrapping.
Starting with NestJS
In experimenting with this solution, I found it much easier to add SAM support to an existing NestJS so that’s what we’ll do first.
To get started, grab the NestJS CLI:
npm i -g @nestjs/cli
With the CLI installed, we can create a new project with:
nest new serverless-nest
You’ll be prompted for which package manager to use. I chose npm.
Open the project in Visual Studio Code
By default the Nest project is bootstrapped as a server that runs and listens on port 3000 for requests.
Requests to our lambda function will be coming from AWS API Gateway, so we don’t need this server to be running. We just want to route the request straight into Nest so we can take advantage of its benefits.
We’re going to change src/main.ts in order to do this:
import { NestFactory } from '@nestjs/core';
import { AppModule } from './app.module';
import { AppService } from './app.service';
async function bootstrap() {
const app = await NestFactory.createApplicationContext(AppModule);
return app;
}
export async function handler(event, context) {
const app = await bootstrap();
const appService = app.get(AppService);
const data = appService.getHello();
return {
statusCode: 200,
body: JSON.stringify({
message: data,
}),
};
}
We still have a bootstrap function but instead of spinning up a server, we’re calling createApplicationContext to get access to the application itself.
Our lambda function handler calls this bootstrap function and in turn calls a getHello() method on a simple service, returning a string.
We stringify that string and return an object with a successful status code.
Once the project is open in Visual Studio Code (or your preferred editor) we can start to add in the files related to AWS SAM.
Arguably the most important file is the template.yaml file which describes your project and tells AWS what resources are required to make your code run when deployed.
# This is the SAM template that represents the architecture of your serverless application
# https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-template-basics.html
# The AWSTemplateFormatVersion identifies the capabilities of the template
# https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/format-version-structure.html
AWSTemplateFormatVersion: 2010-09-09
Description: >
serverless-nest
# Transform section specifies one or more macros that AWS CloudFormation uses to process your template
# https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/transform-section-structure.html
Transform:
- AWS::Serverless-2016-10-31
Globals:
Function:
Layers:
- !Ref RuntimeDependenciesLayer
Runtime: nodejs14.x
MemorySize: 128
Timeout: 100
# Resources declares the AWS resources that you want to include in the stack
# https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/resources-section-structure.html
Resources:
# Each Lambda function is defined by properties:
# https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
# This is a Lambda function config associated with the source code: in src/handlers/example.ts
ExampleFunction:
Type: AWS::Serverless::Function
Metadata:
BuildMethod: makefile
Properties:
Handler: dist/main.handler
Description: A simple example includes a HTTP get method to get all items from a DynamoDB table.
Events:
Api:
Type: Api
Properties:
Path: /
Method: GET
# Shared layer with Lambda runtime dependencies
RuntimeDependenciesLayer:
Type: AWS::Serverless::LayerVersion
Metadata:
BuildMethod: makefile
Properties:
Description: Runtime dependencies for Lambdas
ContentUri: ./
CompatibleRuntimes:
- nodejs14.x
RetentionPolicy: Retain
Outputs:
WebEndpoint:
Description: "API Gateway endpoint URL for Prod stage"
Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/"
This configuration declares two main resources: the lambda function, and the lambda layer that will contain all of the node_modules required. This lambda layer has the benefit of being reusable across other functions in the future.
Custom Build Step
The next file we need is a Makefile. The Makefile is reference in the template.yaml above and is used when building both the lambda function and the lambda layer.
.PHONY: build-RuntimeDependenciesLayer build-lambda-common
.PHONY: build-ExampleFunction
build-ExampleFunction:
$(MAKE) HANDLER=src/main.ts build-lambda-common
build-lambda-common:
npm install
rm -rf dist
nest build
cp -r dist "$(ARTIFACTS_DIR)/"
build-RuntimeDependenciesLayer:
mkdir -p "$(ARTIFACTS_DIR)/nodejs"
cp package.json package-lock.json "$(ARTIFACTS_DIR)/nodejs/"
npm install --production --prefix "$(ARTIFACTS_DIR)/nodejs/"
rm "$(ARTIFACTS_DIR)/nodejs/package.json"
When the SAM CLI builds the project, it will execute these build steps. In the case of build-lambda-common it will clean the previous dist folder before running nest build - transpiling the TypeScript into Javascript. The resulting content of the dist folder are copied to the artifacts folder.
The artifacts folder is a folder created during the SAM CLI build process. The contents of the artifacts folder is copied to S3 and is used during the deployment.
In the case of build-RuntimeDependenciesLayer just the production node_modules are copied into the artifacts folder.
Installing AWS SAM CLI
Now it’s time to installed the AWS SAM CLI tool itself.
The AWS SAM CLI has some prerequisites if you want to test your lambda function locally, so make sure you have Docker installed.
Install the CLI on Mac OS with:
brew tap aws/tap
brew install aws-sam-cli
Check which version you have with:
sam --version
At the time of writing, I was using version 1.22.0.
Installation instructions for other platforms can be found here: https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html
First Run
At this point we can run two commands and see some output.
nest build
sam local invoke
The first command builds the nest project and places the resulting files in the dist folder.
The second command invokes the lambda function through the SAM CLI tool. This doesn’t do any building, but uses the template.yaml to determine how to execute the function, i.e. look for dist/main.handler.
Invoking dist/main.handler (nodejs14.x)
RuntimeDependenciesLayer is a local Layer in the template
Building image.............
Skip pulling image and use local one: samcli/lambda:nodejs14.x-f88a901bbe51b578e9260ae49.
Mounting /Users/adambarker/Documents/Projects/temp/serverless-nest as /var/task:ro,delegated inside runtime container
START RequestId: 5756cb77-72cc-4c75-a991-d377ec7e4418 Version: $LATEST
[Nest] 17 - 04/23/2021, 12:13:34 AM [NestFactory] Starting Nest application...
[Nest] 17 - 04/23/2021, 12:13:34 AM [InstanceLoader] AppModule dependencies initialized +240ms
END RequestId: 5756cb77-72cc-4c75-a991-d377ec7e4418
REPORT RequestId: 5756cb77-72cc-4c75-a991-d377ec7e4418 Init Duration: 2.39 ms Duration: 3582.71 ms Billed Duration: 3600 ms Memory Size: 128 MB Max Memory Used: 128 MB
{"statusCode":200,"body":"{\"message\":\"Hello World!\"}"}
The invocation will take a few seconds as, behind the scenes, an AWS Lambda compatible image is being download to Docker. Your code is then mounted into the Docker container before execution.
Testing the API Gateway Endpoint locally
Another facility the SAM CLI offers is the creation of an endpoint that allows you to send requests to test your lambda function.
sam local start-api
After a similar invocation as sam local invoke, the SAM CLI tells you the server is listening on localhost:3000. You can now test the lambda function in the browser or with curl:
curl "http://localhost:3000"
Making code changes
The start-api command helpfully informs that code changes are now reflected instantly/automatically. It’s watching for changes to the dist folder, which of course is only updated when nest build is run.
We can open a new terminal window and run:
nest build --watch
…which will watch for changes and rebuild our nest project accordingly.
If we change the getHello() method to return a different string:
@Injectable()
export class AppService {
getHello(): string {
return 'Hello Mars!';
}
}
…the project will be rebuilt and we can test again with the curl command.
Speeding things up
You may have noticed that it’s actually quite slow to make iterative changes. One problem I found is that the lambda layer container image is being rebuilt with every invocation of the lambda function.
This is far from ideal. There is some documentation about running sam local start-api with the --warm-containers EAGER option but I was unsuccessful in having this update as I made code changes.
One solution I found is to zip up the node_modules and reference the zip file in the Makefile instead of running a the lambda layer build step each time.
To do this, stop the sam local start-api running in the terminal, and run the sam build command:
sam build
This builds the lambda function and the lambda layer and copies the results to an artifacts directory under .aws-sam/build.
If we change to the .aws-sam/build/RuntimeDependenciesLayer folder we can zip up the contents:
zip -r nm.zip nodejs
We should then move the resulting nm.zip file to the root folder of the project, and remove the .aws-sam folder completely.
With nm.zip in the root, we can modify the template.yaml to use this zip file instead of rebuilding the lambda layer:
...
# Shared layer with Lambda runtime dependencies
RuntimeDependenciesLayer:
Type: AWS::Serverless::LayerVersion
Metadata:
BuildMethod: makefile
Properties:
Description: Runtime dependencies for Lambdas
ContentUri: ./nm.zip
# ContentUri: ./
CompatibleRuntimes:
- nodejs14.x
RetentionPolicy: Retain
...
With the template.yaml updated, we can again run sam local start-api and use curl to test the invocation. After the first run, subsequent invocations should be substantially quicker.
You may notice now that the slowest part of the process is mounting the volume in the docker container. There is an outstanding issue on Mac OS around the performance of mounted volumes (see https://github.com/docker/for-mac/issues/3677 for more information).
Deployment
Finally, we can deploy to AWS.
sam build # to ensure .aws-sam is up to date
sam deploy
This command creates the stack on AWS CloudFormation and provisions the resources required to deploy the application. In this example, the lambda function is created (along with an execution role) and an API Gateway resource.
If all is successful the output should give you the API Gateway endpoint to use to test your deployed lambda function.
