Serverless TypeScript: A complete setup for AWS SAM Lambdas

Serverless functions (or Lambdas in AWS-speak) are a great choice when your application load tends to be highly irregular, and you want to avoid provisioning virtual servers and setting up the full environment to perform some resource-expensive operations for a few days or weeks a year.

Recently, we had a perfect use case for Lambdas on our project with 2U, Inc., a global leader in education technology, as tasks like supporting student admissions or processing exam results are highly seasonal and can be easily handled in a serverless environment.

There are many tools to simplify the development of serverless functions: cross-vendor Serverless framework, AWS-specific Serverless Application Model (SAM), and others.

AWS SAM is a great choice if you already rely on the Amazon Web Services ecosystem. SAM makes it easy to deploy your Lambdas along with all the required cloud infrastructure (API gateways, databases, queues, logging, etc.) with the help of widely used AWS CloudFormation templates.

Even though AWS Lambda supports many runtimes, Node.js remains my number one choice for the richness of its npm ecosystem, the abundance of online documentation and examples, and stellar startup times. However, writing pure JavaScript is arguably less pleasant than running pure JavaScript. Luckily, it’s 2021, and we can rely on TypeScript to bring the joy back into writing JS.

There is just one problem: AWS SAM does not support TypeScript out of the box.

It is a major drawback, but not a reason to give up and throw your types and compile-time checking out of the window. Let’s see how we can build a pleasant SAM-TypeScript experience ourselves: from local development to deployment. And we will not resort to widely recommended hacks like moving your package.json around the project. Stay tuned!

The TypeScript Bingo

In my perfect imaginary world, this is how proper TypeScript support should look like:

• Keeping the local development experience mostly unchanged: no moving package.json to other places or otherwise changing the directory structure before deployment.
• No running sam build on every change in function handler code.
• Keeping generated JS code as close to TS source as possible, preserving the file layout (don’t bundle everything into a single file as webpack does).
• Keeping dependencies in a separate layer shared between related Lambdas. It makes deploys faster as you only need to update function code and not its dependencies. Also, Lambda functions have a size limit which can be easily surpassed with heavy dependencies; shared layers allow us to keep coloring between the lines.
• Keeping deploys as vanilla as possible: sam build and sam deploy, with no extra CLI magic.

In short, a Lambda with TypeScript and shared layers must behave the same way as a freshly generated Lambda on pure Node.js.

Let’s see how we can achieve this!

1. Move dependencies to shared layers

I have reviewed a few manuals on “how to move Node.js dependencies to Lambda layers” (1, 2), but I didn’t follow through on any of them as they propose moving package.json from the root of the project to the required dependencies folder and thus break local development and testing.

Then I stumbled upon the Building layers official doc from AWS and decided to replace a default SAM build process for Node.js runtime (sam build copies code automagically, installs packages, and cleans up, but you can’t interfere with its decisions) with a custom one, based on a Makefile.

First, we declare our own and any third-party layers inside the template.yml:

Globals:
  Function:
    Layers:
      # Our own layer that we are going to build
      - !Ref RuntimeDependenciesLayer
      # At the same time we can also reference third-party layers
      - !Sub "arn:${AWS::Partition}:lambda:${AWS::Region}:464622532012:layer:Datadog-Node14-x:48"

  RuntimeDependenciesLayer:
    Type: AWS::Serverless::LayerVersion
    Metadata:
      BuildMethod: makefile # This does the trick
    Properties:
      Description: Runtime dependencies for Lambdas
      ContentUri: ./
      CompatibleRuntimes:
        - nodejs14.x
      RetentionPolicy: Retain

This Metadata section is a key thing here. We add it not only to our layer but also to our Lambdas:

Metadata:
  BuildMethod: makefile

We can then write a simple Makefile that keeps only the runnable code inside a Lamda itself and puts all the node_modules dependencies inside a separate layer that can be shared with other lambdas.

.PHONY: build-ExampleLambda build-RuntimeDependenciesLayer

build-ExampleLambda:
	cp -r src "$(ARTIFACTS_DIR)/"

build-RuntimeDependenciesLayer:
	mkdir -p "$(ARTIFACTS_DIR)/nodejs"
	cp package.json package-lock.json "$(ARTIFACTS_DIR)/nodejs/"
	npm install --production --prefix "$(ARTIFACTS_DIR)/nodejs/"
	rm "$(ARTIFACTS_DIR)/nodejs/package.json" # to avoid rebuilding when changes don't relate to dependencies

Tip: if you’re using Yarn, then you need --cwd switch instead of npm’s --prefix.

Now there is much less magic and so much more observability in Lambda build process!

However, a Makefile-based build also has its disadvantages: it is hard to debug your build process. See aws/aws-sam-cli#2006 for details.

See this commit in our example template repository on GitHub for more details.

2. Migrate to TypeScript

Now, as we have a customizable build pipeline in place, we can finally add the TypeScript compilation step.

First, let’s install TypeScript itself. Place the following packages into your package.json and define some scripts to compile your TypeScript for development and production:

"dependencies": {
  "source-map-support": "^0.5.19"
},
"devDependencies": {
  "@tsconfig/node14": "^1.0.0",
  "@types/aws-lambda": "^8.10.72",
  "@types/node": "^14.14.26",
  "typescript": "^4.1.5"
},
"scripts": {
    "build": "node_modules/typescript/bin/tsc",
    "watch": "node_modules/typescript/bin/tsc -w --preserveWatchOutput"
}

Second, create and configure your tsconfig.json:

{
  "extends": "@tsconfig/node14/tsconfig.json",
  "compilerOptions": {
    "sourceMap": true,
    "moduleResolution": "node",
    "outDir": "./dist",
    "rootDir": "./src"
  },
  "include": ["src/**/*.ts", "src/**/*.js"]
}

As AWS Lambda nodejs14.x runtime works (obviously) on the latest LTS version of Node.js, we can use "target": "es2020" and "lib": ["es2020"] options to build JS code that will be very, very similar to the source TypeScript code, keeping all the asyncs and awaits.

Now you can replace your build-ExampleLamda target from a Makefile with the definition that includes install and build steps:

# Makefile
build-ExampleLambda:
	npm install
	npm run build
	cp -r dist "$(ARTIFACTS_DIR)/"

3. Drop sam build in local development

Commands like sam local invoke or sam local start-api first look into .aws-sam/ folder, and, if there is none, search for handler code in the current folder.

It is important to remove autogenerated .aws-sam directory after every deployment so SAM can see your local changes without re-running sam build constantly.

We just need to ensure that the compiled TypeScript code is located on the same path both in deployed Lambda and locally.

By default, the Node.js code for a Lamda is located in the src/ folder, but it now contains our TypeScript code, so we need to put our compiled code somewhere else. Let’s borrow a popular convention from the frontend folk and introduce the dist folder for the resulting JavaScript. Change your template.yml:

--- a/template.yml
+++ b/template.yml
@@ -29,7 +29,7 @@ Resources:
     Metadata:
       BuildMethod: makefile
     Properties:
-      Handler: src/handlers/get-all-items.getAllItemsHandler
+      Handler: dist/handlers/get-all-items.getAllItemsHandler

All you need now is to start your TypeScript compiler in watch mode, so your code will magically compile on each change (or not, but then you will immediately know why). Luckily, we have already taken care of this in our package.json. Just don’t forget to run this in your terminal before starting to code (or launch a watch task in your favorite IDE).

$ npm run watch

You don’t need to run sam build locally anymore, and commands like sam local start-api will be able to see your changes immediately (because they’re pointed to transpiled code, not source).

And if you have a Procfile launcher like Overmind (Martian quality, highly recommended!), you can set up starting both SAM and TS compiler in parallel in a Procfile:

# Procfile
sam: sam local start-api
tsc: npm run watch

And then use it to start both TypeScript compiler in watch mode and a local API gateway as two concurrent processes:

$ overmind start

And that’s it!

See a summary of changes we have made in this step in this commit in our example repo.

4. Type up your code

It is also a good idea to enable source maps support, so we’ll get proper TypeScript stack traces in logs in case of any error:

import "source-map-support/register";

We also need to include AWS-specific typings into our handlers:

import { APIGatewayProxyEvent, APIGatewayProxyResult } from "aws-lambda";

And declare handlers using them:

export const getAllItemsHandler = async (
    event: APIGatewayProxyEvent,
): Promise<APIGatewayProxyResult> => {

See the full example of a properly typed test code in this example commit.

5. Set up tests

1. Add Jest with TypeScript support to your package.json:

   "devDependencies": {
     "@types/jest": "^26.0.20",
     "jest": "^26.6.3",
     "ts-jest": "^26.5.1",
   },

2. Add TypeScript-related configuration to your jest.config.js

   module.exports = {
     preset: "ts-jest",
     modulePathIgnorePatterns: ["<rootDir>/.aws-sam"],
   };

And that’s it! Rewrite your tests in TS and run them with npm t, as you did before.

6. Debug

With this approach, debugging is not only possible, but it also works right out of the box!

You can debug with an external debugger following this AWS manual: Step-through debugging Node.js functions locally.

1. Run sam local invoke with the --debug-port option.

$ sam local invoke getAllItemsFunction --event events/event-get-all-items.json --debug-port 5858

That will wait for a debugger to attach before starting the execution of the function.

2. Place a breakpoint where needed (yes, right in your TypeScript code!)

3. Start external debugger (in Visual Studio Code, you can just press F5).

And here is example of .vscode/launch.json for VS Code:

{
  "version": "0.2.0",
  "configurations": [
    {
      "name": "Attach to SAM CLI",
      "type": "node",
      "request": "attach",
      "address": "localhost",
      "port": 5858,
      "localRoot": "${workspaceRoot}/",
      "remoteRoot": "/var/task",
      "protocol": "inspector",
      "stopOnEntry": false
    }
  ]
}

Bonus: Put only relevant code into your Lambdas functions

While it is convenient to have many related Lambda functions together in one SAM project with common configuration, dependencies, and common utility code (reused by some but not all functions), it seems redundant to redeploy all functions at once when you change a little bit of code that is used only by one of them.

I want each given function to use only those files that are really used by this function, no extraneous imports. That is my pure aesthetical maximalism as it doesn’t hurt to put all the files into every function: source code files are pretty lightweight. But TypeScript allows us to achieve purity! When the TypeScript compiler is given a path to a single file, it will only compile this file and its dependencies, nothing else.

Given that every Lambda function has a handler (a single entry point function in a single file), we can compile only this handler, and this will get us only the files required to run a given Lambda.

That allows us to save on source code lines and (more importantly) to redeploy only the functions that use the changed code.

However, to make TSC honor our configuration from tsconfig.json we need a little hack (see microsoft/TypeScript#27379 (comment) for details).

Here it comes:

build-lambda-common:
	npm install
	rm -rf dist
	echo "{\"extends\": \"./tsconfig.json\", \"include\": [\"${HANDLER}\"] }" > tsconfig-only-handler.json
	npm run build -- --build tsconfig-only-handler.json
	cp -r dist "$(ARTIFACTS_DIR)/"

build-getAllItemsFunction:
	$(MAKE) HANDLER=src/handlers/get-all-items.ts build-lambda-common
build-getByIdFunction:
	$(MAKE) HANDLER=src/handlers/get-by-id.ts build-lambda-common
build-putItemFunction:
	$(MAKE) HANDLER=src/handlers/put-item.ts build-lambda-common

And now for the following file structure of our project:

.
└── src
    ├── handlers
    │   ├── a.ts
    │   ├── b.ts
    │   └── c.ts
    └── utils
        ├── ab.ts
        └── bc.ts

We will get the following three lambda functions:

.aws-sam/build/FunctionA
└── dist
    ├── handlers
    │   ├── a.js
    │   └── a.js.map
    └── utils
        ├── ac.js
        └── ac.js.map

.aws-sam/build/FunctionB
└── dist
    ├── handlers
    │   ├── b.js
    │   └── b.js.map
    └── utils
        ├── ab.js
        ├── ab.js.map
        ├── bc.js
        └── bc.js.map

.aws-sam/build/FunctionC
└── dist
    ├── handlers
    │   ├── c.js
    │   └── c.js.map
    └── utils
        ├── bc.js
        └── bc.js.map

If we change only the src/handlers/a.ts, only the FunctionA will be redeployed. And if we change src/utils/bc.ts (imported from handlers b and c), only FunctionB and FunctionC will be redeployed. Neat?

See the full commit here.

In summary

• We need to run tsc -w when we code, but, well, it’s inevitable and makes life more fun.
• Our tests are working just fine.
• Our Lambdas are as small as possible: node_modules folder is inside the external shared layer, and each Lambda includes only the actual code it needs to function (get the pun?).
• We can place breakpoints right in a TypeScript code.
• We’re not reinventing SAM but configuring it to fit our needs.
• Deploy procedure hasn’t changed at all!

This setup isn’t ideal, but it fits our needs pretty well. If you have anything to add, please mention @evilmartians in a tweet (or just open a PR in the example repo).

Show me the code!

If you are new to Lamdas, we give you the fully configured SAM application template with API gateway, database, queues, everything, along with some demo CRUD serverless functions. All you need to give it a go is an AWS account and…

gh repo clone Envek/aws-sam-typescript-layers-example
sam build
sam deploy --guided

Check out the individual commits to understand things better.

If you just want to start developing your Lambdas with this—here’s the template for spinning things up:

sam init --location gh:Envek/cookiecutter-aws-sam-typescript-layers

And you’re ready to deploy example Lambda with TypeScript. Neat?

If you feel like your company can benefit from serverless computing and you need some help to set things up, feel free to give Evil Martians a shout, we will gladly send some top-notch engineers your way.