Every post wants an og:image and a twitter:image, because that is what a link to it looks like when someone shares it. Making those by hand is quite a lot of extra work per post, and the posts that don’t get it end up as blank grey rectangles in the preview.

The usual answer is a script in the site repository. That works, but the layout, the fonts and the colours end up welded to the one site it was written for, and the next site starts again from nothing.

Colophon takes the description of the image from the post and the branding from configuration, so the same package can generate images for sites with different branding. The package name comes from the printer’s colophon, which is an emblem a publisher stamps on a finished work.

The image is described in the post

A post declares what its image should look like, including the image template:

---
title: "@kensio/yulin v0.34.2 adds simulated IAM"
slug: npm-kensio-yulin-simulated-iam
meta_img_props:
  template: banner
  title: "@kensio/yulin"
  subtitle: Simulated IAM
  version: "0.34.2"
---

Running the CLI over a content tree writes one PNG per output size next to each post that declares those props, named after the post’s slug:

colophon content --config colophon.config.ts

That gives npm-kensio-yulin-simulated-iam-og.png and npm-kensio-yulin-simulated-iam-square.png in the post’s own directory. Files that already exist are skipped unless the run is passed --overwrite, so regenerating a whole site only recreates the images that are actually missing.

The default output set is one 1200x630 landscape and one 1200x1200 square, which between them cover og:image and both Twitter card types. Other sizes are available as presets, or as any { name, width, height } you define.

There are three templates so far. banner is a left-aligned title with an optional version, subtitle, corner badge and footer. card is a centred title and subtitle with nothing else. code renders a snippet.

Code images

The code template takes the snippet from the frontmatter and highlights it with real editor theme colours, using Shiki for the grammars and themes:

---
title: eslint changed TypeScript files only
slug: eslint-changed-ts-files-only
meta_img_props:
  template: code
  language: bash
  code: |
    mapfile -t CHANGED_TS < <(
      git diff origin/main --name-only \
        | grep '\.ts'
    )
---

Fitting arbitrary code into a fixed image is the part that needed the most thought. Colophon measures the longest line and the line count against a monospace grid, then picks the largest font size that fits on both axes.

Those bounds are fractions of the image width rather than absolute sizes, because that is what a feed scales a share image to. Sizing against the height instead would render the same snippet in a landscape image at half the size of its square counterpart.

A snippet that still doesn’t fit at the smallest allowed size is truncated with an ellipsis rather than shrunk into something illegible. Since a finished image gives no sign that it lost anything, Colophon reports it:

colophon: content/post/index.md: code snippet does not fit the 1200x630 image at
a legible size: 4 of 13 lines dropped. Shorten the sample, or lower
code.minFontScale to fit it in smaller.

The practical budget at the default settings is around nine lines of about sixty characters for a landscape image. Snippets written for that will render identically at every size.

Branding comes from config

Colours, background, fonts, footer and badge are configuration rather than anything read from the site’s own stylesheet:

import { defineConfig } from "@kensio/colophon";

export default defineConfig({
  colors: { brand: "#2563eb", brandDark: "#1e3a8a", brandWarm: "#f59e0b" },
  footer: "example.com",
  badge: { text: "npm" },
});

Custom templates go in the same config. A template is a name and a render function returning SVG, and anything registered there merges over the built-ins.

For programmatic use, renderMetaImages takes props and config and returns the rendered bytes with no filesystem involved, while generate ties walking a content tree, rendering and writing together in the way the CLI does.

This site runs on it

The images on this blog are generated by a script that is now about twenty lines, most of which is logging what it wrote:

import { generate } from "@kensio/colophon";

import { colophonConfig } from "./image/colophon-config.js";
import { repoPath } from "./util/path.js";

const results = await generate({
  contentDir: repoPath("content"),
  config: colophonConfig(),
  overwrite: process.argv.slice(2).includes("--overwrite"),
});

The config which that script passes to colophon compiles the theme’s own Sass variables and reads the brand colour back out of the result. That means the meta images track the site palette instead of holding a second copy of it.

That replaced a bespoke image generator of a little over three hundred lines living in this repository, which did the same job for this site alone.

What it doesn’t do

Colophon renders SVG to PNG from a small set of layouts. It’s not a design tool. If you need a genuinely different layout, then you can write a new custom template.

It also assumes a static site with frontmatter in markdown files. The render core has no filesystem concerns at all, so it can be driven from something else, but the walker and the CLI are built around that arrangement.

The package is on npm as @kensio/colophon.

Functions are created and invoked entirely in-process and in memory, with no containers and no real AWS infrastructure.

The simulator supports CreateFunctionCommand, GetFunctionCommand and InvokeCommand, including the RequestResponse, Event and DryRun invocation types.

It also supports AWS::Lambda::Function resources when deploying CloudFormation or CDK templates into the simulated AWS.

Function code can come from three places: a real in-process handler function, zip archive bytes on Code.ZipFile, or a zip object stored in sim S3.

The quickest of those is passing an ordinary function from your own process as the function code. It can be stepped through in a debugger and can close over local test state:

import { CreateFunctionCommand, InvokeCommand } from "@aws-sdk/client-lambda";

import { SimAws } from "@kensio/yulin";
import { makeLambdaZipFileInput } from "@kensio/yulin/lambda";

const simAws = new SimAws();
const lambda = simAws.lambda();

await lambda.createFunction(
  new CreateFunctionCommand({
    FunctionName: "example-greeter",
    Role: "arn:aws:iam::123456789012:role/ExampleGreeterRole",
    Code: {
      ZipFile: makeLambdaZipFileInput(
        (event: { name: string }) => `Hello ${event.name}`,
      ),
    },
  }),
);

const output = await lambda.invoke(
  new InvokeCommand({
    FunctionName: "example-greeter",
    Payload: JSON.stringify({ name: "Yulin" }),
  }),
);

console.log(Buffer.from(output.Payload).toString());

makeLambdaZipFileInput produces something the SDK-shaped Code.ZipFile input accepts, so the create call keeps the shape it has against the real AWS SDK.

Handlers take the real (event, context, callback) signature, and typed handlers written against the aws-lambda typings package can be passed in unchanged.

Creating a function requires an execution Role ARN, as on real AWS. A new function starts Pending and becomes Active in the background, so simAws.backgroundTasksComplete() is there for tests that assert on the state.

A handler that throws is reported AWS-style, with FunctionError: "Unhandled" and an error document payload, rather than the invoke call itself throwing.

Real zip archives work too. makeLambdaCodeZip builds real zip bytes from a source string or from a files map keyed by archive path.

That archive runs in a Node.js vm context with cold start semantics, so the module is imported once on first invocation and its state stays warm across invocations afterwards.

The runtime provides @aws-sdk/* packages the way the real Node.js runtime does, without them being bundled into the archive. Clients constructed inside function code are routed into the same simulated AWS environment.

Calls the handler makes run as the function’s execution role, so simulated IAM authorizes them just as a real execution role would. Given an execution role allowed to read from example-bucket:

import { CreateFunctionCommand, InvokeCommand } from "@aws-sdk/client-lambda";

import { SimAws } from "@kensio/yulin";
import { makeLambdaCodeZip } from "@kensio/yulin/lambda";

const simAws = new SimAws();

await simAws.lambda().createFunction(
  new CreateFunctionCommand({
    FunctionName: "example-reader",
    Role: readerRole.Arn,
    Handler: "index.handler",
    Code: {
      ZipFile: makeLambdaCodeZip(`
        const { S3Client, GetObjectCommand } = require("@aws-sdk/client-s3");
        const s3Client = new S3Client({});
        exports.handler = async (event) => {
          const output = await s3Client.send(
            new GetObjectCommand({
              Bucket: "example-bucket",
              Key: event.objectKey,
            }),
          );
          return await output.Body.transformToString();
        };
      `),
    },
  }),
);

const output = await simAws.lambda().invoke(
  new InvokeCommand({
    FunctionName: "example-reader",
    Payload: JSON.stringify({ objectKey: "public/config.json" }),
  }),
);

The require("@aws-sdk/client-s3") there resolves to the real package, with its clients pointed at the simulated AWS rather than at anything on the network.

If the execution role is not allowed to make that read, sim IAM denies it and the invocation reports the denial as a function error, in the same way a real execution role denial surfaces inside the handler.

Sim CloudFormation can create functions as well, so they can come from the same template as the rest of the infrastructure:

{
  "Resources": {
    "ExampleGreeterFunction": {
      "Type": "AWS::Lambda::Function",
      "Properties": {
        "FunctionName": "example-greeter",
        "Role": "arn:aws:iam::123456789012:role/ExampleGreeterRole",
        "Handler": "index.handler",
        "Runtime": "nodejs22.x",
        "Code": {
          "ZipFile": "exports.handler = async (event) => 'Hello ' + event.name;"
        }
      }
    }
  }
}

Inline ZipFile template source is packaged and run in the vm runtime, as if it had been zipped and passed to CreateFunctionCommand.

The Role is usually an Fn::GetAtt reference to a same-stack AWS::IAM::Role, which resolves to that Role’s ARN. Ref on the function returns its name and Fn::GetAtt supports Arn.

Deploy-time bindings can back a template function with a real in-process handler instead of its template code, which is the CloudFormation counterpart of makeLambdaZipFileInput:

import { SimAws } from "@kensio/yulin";

const simAws = new SimAws();

await simAws.cloudFormation().deployTemplate({
  stackName: "example-greeter-stack",
  template: exampleTemplate,
  bindings: [
    {
      logicalId: "ExampleGreeterFunction",
      handler: (event: { name: string }): string => `Hello ${event.name}`,
    },
  ],
});

A bound function can leave out its template Code and Handler entirely, and other functions in the same template keep running their template code.

Bindings can target a function by logicalId, functionName, arn or full cdkPath, and the logical ID also matches a CDK construct ID from aws:cdk:path metadata.

Sim Lambda covers what isolated tests and local development need rather than full Lambda parity. The Lambda tests in the repository are the clearest picture of what is actually covered.

UpdateFunctionCode, DeleteFunction and function listing are not implemented yet, and versions, aliases and qualifiers are not simulated.

The vm runtime runs CommonJS function code only, so ES module source fails with a hint rather than running. Container image functions are not supported, as the simulator stays Docker-free, and Layers are not simulated.

Environment.Variables is not applied yet, and Timeout is recorded without interrupting the handler.

The vm context is a namespacing convenience rather than a security boundary. Function code runs in-process with the same trust as the test suite itself, so this is not a way to run untrusted code.

The Lambda simulator integrates with the rest of Yulin, so S3, IAM, CloudFormation and CDK can all participate in the same simulated environment for local development and system tests.

npm: https://www.npmjs.com/package/@kensio/yulin

The simulator supports Roles with CreateRoleCommand and Users with CreateUserCommand, plus inline policies with PutRolePolicyCommand and PutUserPolicyCommand.

Managed policies are created with CreatePolicyCommand and attached with AttachRolePolicyCommand.

Sim IAM evaluates those policies into allow/deny decisions. Simulated STS issues temporary Role sessions with AssumeRoleCommand, after checking the Role’s trust policy.

Creating a Role with an inline policy and authorizing an action against it looks much the same as it does against the real AWS SDK:

import { CreateRoleCommand, PutRolePolicyCommand } from "@aws-sdk/client-iam";
import { SimAws } from "@kensio/yulin";

const simAws = new SimAws();
const simIam = simAws.account("123456789012").iam();

const createRoleOutput = await simIam.createRole(
  new CreateRoleCommand({
    RoleName: "ExampleReaderRole",
    AssumeRolePolicyDocument: JSON.stringify({
      Version: "2012-10-17",
      Statement: {
        Effect: "Allow",
        Principal: { AWS: "arn:aws:iam::123456789012:root" },
        Action: "sts:AssumeRole",
      },
    }),
  }),
);

await simIam.putRolePolicy(
  new PutRolePolicyCommand({
    RoleName: "ExampleReaderRole",
    PolicyName: "ReadExampleObjects",
    PolicyDocument: JSON.stringify({
      Version: "2012-10-17",
      Statement: [
        {
          Effect: "Allow",
          Action: "s3:GetObject",
          Resource: "arn:aws:s3:::example-bucket/*",
        },
        {
          Effect: "Deny",
          Action: "s3:GetObject",
          Resource: "arn:aws:s3:::example-bucket/protected/config.json",
        },
      ],
    }),
  }),
);

const decision = simIam.authorize({
  action: "s3:GetObject",
  resource: "arn:aws:s3:::example-bucket/public/config.json",
  caller: { kind: "arn", arn: createRoleOutput.Role.Arn },
});

console.log(decision.isAllowed);

The authorize() method returns a decision object rather than throwing, so a test can assert on why a request was allowed or denied.

The decision exposes value as "Allow", "ExplicitDeny" or "ImplicitDeny", along with the statements that matched and the resolved caller.

An explicit Deny in any evaluated policy wins. Otherwise a matching Allow in an identity or resource policy allows the request, and anything else is an implicit deny.

The part that matters most for system tests is that the other simulated services authorize their own actions through sim IAM.

That covers S3, Route53, DynamoDB, ACM, CloudFront and Lambda actions, including s3:GetObject, route53:CreateHostedZone, dynamodb:PutItem and lambda:InvokeFunction.

Sim IAM and sim CloudFormation authorize their own control plane commands as well, so iam:CreateRole and cloudformation:CreateStack can be denied for a caller that is not allowed to make them. Those commands take an optional caller:

import { GetObjectCommand } from "@aws-sdk/client-s3";

const simS3 = simAws.account("123456789012").region("eu-west-2").s3();

const output = await simS3.getObject(
  new GetObjectCommand({
    Bucket: "example-bucket",
    Key: "public/config.json",
  }),
  {
    caller: { kind: "arn", arn: createRoleOutput.Role.Arn },
  },
);

Reading protected/config.json with that same caller throws an AWS-like access denied error with a 403 status code, before the simulated S3 service looks the Object up at all.

Omitting the caller defaults to the Account root, which is allowed within its own Account, so existing tests that never mention IAM keep working.

S3 Bucket policies are supplied to sim IAM as resource policies for the request, so a Bucket policy can allow or deny a read that identity policies say nothing about.

Policy conditions are supported for the StringEquals, StringLike and NumericLessThanEquals operators, including the ForAllValues: and ForAnyValue: set variants of the string operators. Condition values such as S3 Object tags are supplied by the service handling the request.

Access keys created with CreateAccessKeyCommand are registered with the simulated Account, so a caller can be given as credentials rather than an ARN.

Those credentials are authenticated before any policies are evaluated, as are the temporary session credentials returned by simulated STS.

Sim CloudFormation can create IAM resources too, so Roles and policies can come from the same template as the rest of the infrastructure:

{
  "Resources": {
    "ExampleRole": {
      "Type": "AWS::IAM::Role",
      "Properties": {
        "RoleName": "ExampleReaderRole",
        "AssumeRolePolicyDocument": {
          "Version": "2012-10-17",
          "Statement": {
            "Effect": "Allow",
            "Principal": {
              "Service": "lambda.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
          }
        },
        "Policies": [
          {
            "PolicyName": "ReadExampleObjects",
            "PolicyDocument": {
              "Version": "2012-10-17",
              "Statement": {
                "Effect": "Allow",
                "Action": "s3:GetObject",
                "Resource": "arn:aws:s3:::example-bucket/*"
              }
            }
          }
        ]
      }
    }
  }
}

AWS::IAM::Role, AWS::IAM::ManagedPolicy and AWS::IAM::Policy are supported.

An AWS::IAM::Policy puts its document onto each Role named in Roles as an inline policy, which is the shape CDK grants such as bucket.grantRead(fn) synthesise into.

IAM Users and Groups are not simulated as policy principals, so a template naming them fails deployment rather than dropping the grant.

Sim IAM implements the policy behaviour that multi-service tests most commonly need rather than full IAM parity.

Permissions boundaries, session policies and service control policies are not evaluated. Managed policies have a single version, and deleting or detaching Roles, Users, policies and access keys is not supported yet.

IAM is usually the last part of an AWS system to get any testing. Tests mock the SDK call away, so the policies themselves are only exercised once everything is deployed.

That is a slow way to find out that a Role is missing a permission, or that a wildcard was broader than it looked.

This is why I think IAM is one of the more useful things to simulate. A test can deploy the real template, act as a specific Role, and assert that the read it should not be allowed to make is actually denied, in the same run as the rest of the system tests.

The IAM simulator integrates with the rest of Yulin, so S3, Route53, DynamoDB, CloudFront, Lambda, CloudFormation and CDK can all participate in the same simulated environment for local development and system tests.

How to use the Yulin local AWS simulator for Node.js to simulate website hosting, including Route53, S3, CloudFront and CloudFront Functions, provisioned via simulated CloudFormation.

YouTube: https://www.youtube.com/watch?v=hkhi5uzc03w

Transcript

In this video, I’m going to show you how to simulate AWS for static website hosting entirely on local host, including Route 53, ACM, S3, Cloudfront, and CloudFront Functions.

So this is the demo website project. It’s an Astro website built with the Astro framework. Let’s see what the website looks like.

Here we have the astro dev command, which starts a local development server. I’ll run that and we get the Astro dev server on local host port 4321. Let’s open it.

Here’s the demo website running on local host 4321. It’s just the official Astro blog theme with a few small modifications. So that’s our starting point.

Now, that’s fine if you just want to view the website locally. But what if you’re deploying it to AWS. A production deployment usually involves several AWS services. For example, you might be using CDK, and your stack could look something like this.

This is a real CDK stack for deploying a static website. Lets walk through it. First, we have a domain name. This isn’t a real registered domain. It’s just for the demo, yulin-demo-website.com.

We have a Route 53 hosted zone for that domain, an ACM certificate so that we can use HTTPS instead of HTTP and an S3 bucket containing the static website files.

We also have two CloudFront Functions, which we’ll come back to shortly, and then we have a CloudFront distribution, which actually serves the website over the internet using our domain name and ACM certificate. It also associates those two CloudFront Functions.

Next is an S3 bucket deployment. This is a CDK feature that automatically copies files from a local directory into the S3 buckets during deployment, which is very useful for static websites.

Finally, we have a Route 53 alias record. Alias records are an AWS specific feature that let you point the root of a domain directly at an AWS service, such as a CloudFront Distribution. We also have the IPv6 equivalent.

The only other things in the stack are three outputs which expose useful values, like the website URL, and we’ll use those later.

If we look back at the bucket deployment, it’s expecting to copy files from a local dist directory. Right now, that directory doesn’t exist because it’s produced by the Astro build step. So let’s run the build.

Cool. That builds the website instead of just serving it. We can now see the generated HTML files in the dist directory, which means the bucket deployment now has something to upload.

The other thing we need is a CloudFormation template. CDK generates this for us and it’s what we use to deploy to either real AWS or our local simulation. I’ll run pnpm exec cdk synth that prints the template to the terminal, but it also creates a new cdk.out directory containing the deployment artefacts.

Here’s the JSON template. It’s the CloudFormation version of our CDK stack and describes all of the resources and their configuration.

Now that we have that, we’re ready to simulate the entire stack locally with Yulin. Below the Astro scripts, we have another script called local-dev.mts. Let’s open that.

This creates a simulated AWS environment and starts a local host server that serves it. It then uses simulated CloudFormation to deploy the JSON template that we just generated. We point it at the template inside cdk.out and it deploys all of those resources into the simulated AWS environment. Finally, it reads the website URL output from the stack, asks the local server for the local host equivalent and logs that URL for convenience. So let’s run it.

Cool. That’s deployed the entire stack into simulated AWS. The first thing you’ll notice is that we have our real domain name with the Route 53 hosted zone attached as a prefix to the sim-aws.localhost domain. This works not only for Route 53 hosted zones, but also for CloudFront distribution domain names, and S3 website host names.

Let’s open the simulated website. The browser is now visiting our simulated Route 53 domain, and the website looks identical to the one served by Astro’s development server.

If we open the developer tools and reload the page, let’s have a look at the response headers, you’ll notice a few extra security headers, such as Referrer-Policy and X-Content-Type-Options.

Now let’s compare that with the Astro development server. If we reload the same page there, those headers aren’t present, so where are they coming from?

Back in our project, they’re being added by the viewer response CloudFront Function that we saw earlier. A viewer response function intercepts responses on their way back to the client. In our CDK stack, this resource brings the CloudFront Function source code into the deployment and then associates it with the CloudFront distribution as a viewer response function.

Let’s look at the function itself. The source code lives in src/cff/viewer-response. As you can see, this is the code that’s adding those security headers before the response is sent back to the browser. So that’s one example of functionality. We’re able to simulate and test locally.

We also have another CloudFront Function, the viewer request function. It’s associated with the distribution as a viewer request function and its source code lives in viewer-request. Let’s open that one. This function does a bit more.

One thing it handles is trailing slash or pretty URL routing. A URL ending with a slash is actually served by an index.html object in S3 behind the scenes. For example, blog/first-post/ is really serving blog/first-post/index.html. That means visitors only need to browse to blog/first-post/ and this CloudFront Function transparently maps that to the correct file. That behaviour is the same in both real AWS and our local simulation.

The function also handles legacy redirects. For example, maybe this site originally served blog posts under /posts but later we decided /blog was a better URL structure. We can keep those old links working by redirecting /posts to /blog inside the viewer request function.

Let’s try that. If I visit /post, we’re automatically redirected to /blog. If we look in dev tools, the request to /posts returned a 301 Moved Permanently and the Location header points to /blog. That redirect is coming directly from the viewer request CloudFront Function we just looked at.

Now compare that with the Astro development server. If I visit /post there, I just get a 404 because the development server doesn’t know about the production redirect rules. With the simulation, we can test exactly the same redirect behaviour that we’ll have in production on AWS without deploying anything.

So far, we’ve simulated the production behaviour locally. But what if there’s a bug in one of our CloudFront Functions and we want to debug it using our IDE. Without Yulin, that’s usually quite difficult and on many projects, it isn’t really practical. With Yulin, it’s quite straightforward.

Let’s go back to our local dev script. At the moment, all it’s doing is telling the simulator which CloudFormation template to deploy. If we look back at the CDK stack, you’ll notice that each CloudFront Function has its source code embedded directly into the generated JSON template. That’s how both real AWS and the simulator normally execute the function.

But in the simulator, we don’t have to use the embedded source code. Instead, we can use a feature called bindings. Bindings let us replace an embedded CloudFront Function with a real JavaScript function running in the same Node.js process as the simulator.

Let’s add one. We’ll create a bindings array where each entry replaces one resource in the template. First, we specify the logical ID of the resource we want to replace. In this case, it’s the viewer request function. Next, we need the handler. We’ll import that directly from our source code. Now we can pass it into the binding.

So what’s happening here is we’re telling Yulin to find the embedded handler for this CloudFront Function and replace it at runtime with the real handler function we’ve imported from our project. Everything now runs in a single process, which means our IDE debugger can see it.

Let’s stop the simulator and start it again, this time in debug mode. Everything looks much the same so far. The website still behaves exactly as before.

Now let’s go back to the viewer request function and set a breakpoint. I’ll put one here. Now when I refresh the page, the debugger stops on our breakpoint. If we switch back to the browser, it’s waiting for a response because execution is currently paused inside the CloudFront Function.

I’ll remove that breakpoint and continue, the page loads normally again. Earlier, we looked at the legacy redirect, so let’s put a breakpoint there instead. If I visit /post we’re taken straight back into the debugger. From here, we can inspect everything that’s available to the function. We can look at the event, the context, the request, and any other values we need while debugging.

Once we’re done, we simply continue execution, the redirect is returned, the browser follows it, and we end up on the correct /blog page, and all of that is enabled by this small local dev script, only about 28 lines of code.

The simulator supports creating Hosted Zones and record sets through Route53 SDK commands, and also supports AWS::Route53::HostedZone and AWS::Route53::RecordSet resources when deploying CloudFormation or CDK templates into the simulated AWS.

One feature that’s particularly useful for local development and testing is simulated CNAME resolution. When Yulin is running on localhost, sim Route53 records can route your own application hostnames to other simulated AWS services.

For example, you can create a simulated CloudFront distribution, point a Route53 record at it, and then access it locally using a URL like:

http://www.example.test.sim-aws.localhost:3000/

That makes it possible to exercise applications using realistic hostnames while everything still runs locally in a single Node.js process.

Creating a Hosted Zone looks much the same as it does against the real AWS SDK:

const simAws = new SimAws();
const route53 = simAws.route53();

await route53.createHostedZone(
  new CreateHostedZoneCommand({
    Name: "example.test",
    CallerReference: "example-zone",
  }),
);

You can also deploy sim Route53 infrastructure directly from CloudFormation or CDK templates. For example with an output JSON template like:

{
  "Resources": {
    "SiteZone": {
      "Type": "AWS::Route53::HostedZone",
      "Properties": {
        "Name": "example.test"
      }
    },
    "SiteRecord": {
      "Type": "AWS::Route53::RecordSet",
      "Properties": {
        "HostedZoneId": {
          "Ref": "SiteZone"
        },
        "Name": "www.example.test",
        "Type": "CNAME",
        "TTL": "300",
        "ResourceRecords": [
          "d111111abcdef8.cloudfront.net"
        ]
      }
    }
  }
}

You can deploy that into simulated AWS:

import { SimAws } from "@kensio/yulin";

const simAws = new SimAws();

await simAws.cloudFormation().deployTemplateFile("path/to/template.json");

You can then interact with the simulated AWS including sim Route53 on localhost and in automated tests.

The Route53 simulator integrates with the rest of Yulin, so CloudFormation, CDK, S3 and CloudFront can all participate in the same simulated environment for local development and system tests.

Documentation: https://yulinsim.dev/services/route53/

GitHub: https://github.com/KensioSoftware/yulin

npm: https://www.npmjs.com/package/@kensio/yulin

For example, for slightly misused assertions like these:

assertIdentical(fooBool, true);

assertIdentical(fooList.length, 2);

assertTrue(typeof fooStr === "string");

The ESLint rules will highlight each with a warning and a suggestion to use a more appropriate assertion instead:

// ESLint: Use assertTrue(value) instead of assertIdentical(value, true).
// (no-restricted-syntax)
assertTrue(fooBool);

// ESLint: Use a more specific length assertion, such as
// assertArrayLength(value, expectedLength) or assertStringLength(value, expectedLength),
// instead of assertIdentical(value.length, expectedLength). (no-restricted-syntax)
assertArrayLength(fooList, 2);

// ESLint: Use a more specific type assertion, such as assertTypeString(value),
// assertTypeNumber(value), or assertTypeBoolean(value), instead of
// assertTrue(typeof value === expectedType). (no-restricted-syntax)
assertTypeString(fooStr);

This is useful for reminding human developers about better assertion options, and for improving on output from LLMs, which tend to overuse the more general assertion functions.

Using the more specific assertion functions allows a project to get more benefit from the type narrowing effects in the @kensio/smartass library.

Some more examples of the replacements that the ESLint rules will suggest:

• assertIdentical(value, true) → assertTrue(value)
• assertIdentical(value, false) → assertFalse(value)
• assertIdentical(value, undefined) → assertUndefined(value)
• assertTrue(value != null) → assertNonNullable(value)
• assertIdentical(typeof value, "string") → assertTypeString(value)
• assertIdentical(value instanceof MyClass, true) → assertInstanceOf(value, MyClass)
• assertIdentical("key" in object, true) → assertObjectHasProperty(object, "key")
• assertIdentical(value.length, expected) → assertArrayLength() etc.
• assertIdentical(value.size, expected) → assertSetSize() etc.
• assertTrue(array.length > 0) → assertArrayNotEmpty(array)
• assertTrue(value.includes(expected)) → assertStringIncludes() etc.
• assertTrue(value.startsWith(prefix)) → assertStringStartsWith(value, prefix)
• assertTrue(value.endsWith(suffix)) → assertStringEndsWith(value, suffix)
• assertTrue(stats.isDirectory()) → assertDirectoryExists() etc.

You can use the ESLint rules in your project’s eslint.config.ts like this:

import { defineConfig } from "eslint/config";
import tseslint from "typescript-eslint";

import { smartassPreferSpecificAssertions } from "@kensio/smartass/eslint";

export default defineConfig(
  ...tseslint.configs.recommended,
  ...smartassPreferSpecificAssertions,
);

And finally, a quick note on how the rules work internally. The rules use ESLint selector syntax to identify misused assertion functions. For example:

[{
  selector:
    "CallExpression[callee.name='assertIdentical'] > Literal[value=true]:nth-child(2)",
  message:
    "Use assertTrue(value) instead of assertIdentical(value, true).",
}]

• CallExpression — look for a function call
• [callee.name='assertIdentical'] — where the function being called is assertIdentical
• > Literal[value=true] — whose argument is the literal true
• :nth-child(2) — specifically the second argument

Note how this is similar to CSS selector syntax; it works in a similar way.

npm: https://www.npmjs.com/package/@kensio/smartass

GitHub: https://github.com/KensioSoftware/smartass

Docs: https://smartassertions.dev/

This feature allows you to deploy JSON templates from CloudFormation, SAM or CDK into the simulated AWS. Using your existing CloudFormation templates for local development and testing is convenient and efficient.

The simulated CloudFormation deployment only takes a few milliseconds and sets up the simulated AWS resources in the same single process alongside your Node.js application code and tests. You can step through the whole system in a debugger, as well as collect test coverage metrics from the simulated system tests.

Given a basic CloudFormation JSON output template like this:

{
  "Resources": {
    "SiteBucket": {
      "Type": "AWS::S3::Bucket",
      "Properties": {
        "BucketName": "example-site-bucket",
        "WebsiteConfiguration": {
          "IndexDocument": "index.html"
        }
      }
    }
  }
}

You can directly deploy it into the simulated AWS via the sim CloudFormation service:

import { SimAws } from "@kensio/yulin";

const simAws = new SimAws();
const simCfn = simAws.cloudFormation();

const stack = await simCfn.deployTemplate({
  stackName: "site-stack",
  template: {
    Resources: {
      SiteBucket: {
        Type: "AWS::S3::Bucket",
        Properties: {
          BucketName: "example-site-bucket",
          WebsiteConfiguration: {
            IndexDocument: "index.html",
          },
        },
      },
    },
  },
});

await stack.waitForDeployComplete();

That creates the simulated S3 Bucket and configures static website hosting for it according to the configuration in the JSON template.

There is also a deployTemplateFile() convenience method that takes a file path and deploys the JSON template at that path into the simulated AWS:

import { SimAws } from "@kensio/yulin";

const simAws = new SimAws();

const stack = await simAws
  .cloudFormation()
  .deployTemplateFile("foo/bar/path/template.json");

await stack.waitForDeployComplete();

Because the simulated CloudFormation uses normal JSON output templates, it can deploy templates created by standard CloudFormation, SAM or CDK. You can use the existing CloudFormation setup for your project directly with the simulated AWS, making it easy to run realistic system tests and develop locally on your laptop.

By default the simulated CloudFormation deploys CloudFront Functions from the handler source code embedded in the JSON template. This works and supports low-effort configuration. There is also an optional bindings parameter that you can use to swap in the real handler function implementations for your CloudFront Functions when deploying into simulated AWS:

import { SimAws } from "@kensio/yulin";

import { viewerRequestHandler } from "./path/to/viewer-request-handler.js";

const simAws = new SimAws();

const stack = await simAws
  .cloudFormation()
  .deployTemplateFile({
    templatePath: "cdk.out/TestStack.template.json",
    bindings: [
      {
        logicalId: "FooCloudFrontFunction",
        handler: viewerRequestHandler,
      },
    ],
  });

The simulated CloudFormation deployment uses the logicalId of the CloudFront Function to swap in the real handler function in the same process. This allows you to run the whole system simulation and step through it in a debugger, including inside CloudFront Function handler code. It also allows you to collect test coverage metrics for your CloudFront Functions.

Docs: https://yulinsim.dev/services/cloudformation/

GitHub: https://github.com/KensioSoftware/yulin

npm: https://www.npmjs.com/package/@kensio/yulin

CloudFront Functions types in JavaScript / JS2 files:

/**
* @typedef {import("@kensio/yulin/cloudfront").CloudFrontFunction.Event} CloudFrontEvent
* @typedef {import("@kensio/yulin/cloudfront").CloudFrontFunction.Request} CloudFrontRequest
* @typedef {import("@kensio/yulin/cloudfront").CloudFrontFunction.Response} CloudFrontResponse
*/

CloudFront Functions types in TypeScript files:

import { type CloudFrontFunction } from "@kensio/yulin/cloudfront";

const event: CloudFrontFunction.Event = {
  // ...
};

CloudFront Functions JS2 syntax ESLint config in eslint.config.ts:

import { defineConfig } from "eslint/config";
import tseslint from "typescript-eslint";

import { cloudFrontFunctionsJs2 } from "@kensio/yulin/eslint";

export default defineConfig(
  ...tseslint.configs.recommended,
  ...cloudFrontFunctionsJs2,
);

Then you can work on CFF JS2 in .cff.js files with some basic typing and ESLint support.

Docs: https://yulinsim.dev/

GitHub: https://github.com/KensioSoftware/yulin

npm: https://www.npmjs.com/package/@kensio/yulin

The goal is to incorporate existing type information from the calling scope when possible, rather than widening values to a less precise asserted type.

For example, consider a value whose type is already known to be:

import { assertObjectMatches, typeString } from "@kensio/smartass";

interface Foo {
  bar?: {
    foobar?: "hello" | "world" | 123 | null;
  };
}

function getFoo(): Foo {
  return { bar: { foobar: "hello" } };
}

const foo = getFoo();

assertObjectMatches(foo, {
  bar: { foobar: typeString() },
});

foo.bar.foobar;
// "hello" | "world"

The typeString() composable matcher works from within assertObjectMatches() to provide type refinement back to the calling scope. TypeScript can combine this type assertion with the existing type information in the calling scope to precisely narrow the type of foo.bar.foobar to the literal union type "hello" | "world";

Prior to this improvement, it would have been easy for an assertion signature to broaden the type to just string.

Version v1.15.0 updates all the assertion signatures and composable matchers in the package to provide this overlap-preserving type refinement.

This tends to be helpful in tests, because it combines runtime validation with precise compile-time type checking. After an assertion succeeds, TypeScript can often infer more specific types than before, which reduces the need for manual casts and helps IDE autocomplete provide more accurate suggestions.

npm: https://www.npmjs.com/package/@kensio/smartass

GitHub: https://github.com/KensioSoftware/smartass

Docs: https://smartassertions.dev/

This allows you to apply structured assertions that provide detailed type narrowing back to TypeScript:

import {
  assertObjectMatches,
  arrayIncluding,
  oneOf,
  stringOfLength,
} from "@kensio/smartass";

const user = getUser();

assertObjectMatches(user, {
  role: oneOf(["admin", "editor", "viewer"]),
  tags: arrayIncluding("beta"),
  id: stringOfLength(8),
});

// TypeScript knows:
//   user.role is 'admin' | 'editor' | 'viewer'
//   user.tags is an array with at least one string element
//   user.id is a string of length 8 with safe indexing

As with the standalone type narrowing assertion functions, this is made possible by TypeScript’s assertion signature feature.

The fluent expectation interfaces in Jest and Vitest are not able to provide that type narrowing effect back to the compiler, because TypeScript cannot follow an assertion signature through a fluent interface like:

import { expect } from "vitest";

expect(user).toMatchObject({
  role: "editor",
});

Due to the expect().toMatchObject() chain, we lose the chance to provide an assertion signature to TypeScript. It seems like a shame to miss out on the benefit of that type information, so @kensio/smartass now provides composable type narrowing matchers.

The assertObjectMatches function applies a deep partial match, so you only need to specify the properties that a particular test case is interested in.

After calling that assertion function, you can use the narrowed types for the rest of the test. This tends to reduce the amount of boilerplate code that is required in each test case, which makes the tests more readable and easier to maintain.

As of version v1.10.0, the following type narrowing matcher functions are available:

• arrayIncludingAll
• arrayIncluding
• arrayOfLength
• arrayOfMinLength
• nonEmptyArray
• bufferEqualTo
• instanceOf
• nonNullable
• numberBetween
• numberToNearest
• objectWithProperty
• oneOf
• stringEndingWith
• stringIncluding
• stringOfLength
• stringNotIncluding
• stringStartingWith
• typeBigInt
• typeBoolean
• typeFunction
• typeNumber
• typeNumeric
• typeObject
• typeString
• typeSymbol
• typeTypedArray
• uuidV4

These can be composed together in partial structures for precise type narrowing assertions in tests.

npm: https://www.npmjs.com/package/@kensio/smartass

GitHub: https://github.com/KensioSoftware/smartass

Docs: https://smartassertions.dev/

Yulin simulates AWS services entirely in-process, allowing tests and local development environments to interact with realistic AWS behaviour without requiring real AWS infrastructure, Docker containers or any i/o at all.

This differs from traditional test mocking by simulating internal state and overall behaviour. Traditional mocks don’t tend to provide a lot of value, and often end up being detrimental by requiring so much repetitive boilerplate.

The same Yulin configuration can be shared between tests and local development servers, so you can define your AWS simulation once and then use it for both those purposes. Because it all runs in the same single-threaded Node.js process, you can step through the whole system (AWS plus your applications) in the debugger. This also means you can bring in other testing tools like nock HTTP interception and vitest fake timers.

Yulin is still in an early stage and currently only simulates a small subset of AWS services and behaviours. The long-term goal is to allow simulating multiple complex applications that involve several AWS services together, such as S3, CloudFront, SQS, DynamoDB and Lambda.

Simulate, don’t mock

There are already several ways to test AWS-based applications.

At one end of the spectrum are traditional pure unit tests with heavy mocking of dependencies. These are usually fast, but the mocking tends to be fragile and difficult to maintain. A Lambda that writes to DynamoDB might be tested by mocking the DynamoDB client and asserting that a particular method was called with particular arguments. Any time there’s a change related to that, the tests and mocks have to be meticulously updated.

That kind of test can be useful, but they only verify a narrow slice of the overall system behaviour. They also tend to become brittle, as implementation details leak into the tests, and the mocks cannot keep up with ongoing changes to the system design.

At the other end of the spectrum are integration tests running against real AWS infrastructure. In theory, these can provide much stronger confidence, but they are slower, more expensive and require even more setup and maintenance. They are also at least as fragile as pure unit tests with mocks, but in a different way. Integration tests are prone to false positives due to networking issues, cold-starts and asynchronous behaviour.

Tools like LocalStack do bridge the gap between those two extremes, but in practice they often require even more setup and configuration than integration tests that hit real AWS infrastructure.

Many systems would benefit from tests that exercise realistic interactions between AWS services while still remaining fast, isolated and easy to run locally.

Simulating AWS behaviour

Yulin takes the approach of simulating AWS services directly inside the Node.js process, while avoiding magic and clever tricks.

A test can create a simulated AWS environment, configure services and then execute application code against that environment. Everything runs in memory and within the same process.

Because there is no real networking involved, tests remain fast. There are no containers to start, no infrastructure to provision and no external dependencies to coordinate.

This also makes local development easier. The same simulated environment can be used both by automated tests and for running the system locally and interacting with it via localhost.

Testing systems instead of method calls

One benefit of this approach is that it shifts the focus of testing away from implementation details and towards meaningful system behaviour.

Rather than asserting that a particular SDK method was called, a test can exercise a larger workflow and verify the resulting system state.

For example, a test might verify that a Lambda processes an event correctly, updates data in DynamoDB and publishes the expected messages to SQS. The exact implementation details are often less important than whether the overall behaviour is correct.

This kind of testing tends to align more closely with the behaviours that users and stakeholders actually care about.

Local development

As a software engineering contractor, I see a lot of teams struggle or give up with getting their system to run locally on engineers’ laptops. That then leads to contention on development environments in AWS, which are also expensive to operate and maintain.

More importantly, there is a large hidden cost when engineers cannot iterate rapidly on the applications they are building. If deploying to AWS is the only option for properly testing changes, the development iteration cycle gets bogged down, and before long one cycle can take hours or even days. If it’s possible to run the system locally on your laptop, that can instead be minutes or seconds.

To solve that problem, Yulin can expose on localhost the simulated AWS services and the applications running amongst them. This allows other local applications, development tools and browsers to interact with applications in the simulated AWS environment using realistic endpoints and behaviours.

One small example is static website hosting via S3. Yulin can serve a simulated S3 website locally while still supporting S3 website features such as routing rules, redirects and error documents. That is integrated with the rest of the simulation, so you can have a DynamoDB stream trigger a Lambda function that writes an object to an S3 website bucket. You can then view the resulting web page in your browser on localhost.

Using the same configuration for local development and automated tests can help reduce duplication and make development environments more representative of production behaviour.

Early stages

I have to emphasise that Yulin is still a work in progress. Only a small subset of AWS services and behaviours are implemented so far, and Yulin’s APIs will evolve as the project progresses. The focus at this stage is implementing the behaviours that I happen to need for my own projects.

Even in its current minimal state, Yulin is already useful to me for quickly iterating and testing on projects.

Docs:

https://yulinsim.dev/

GitHub:

https://github.com/KensioSoftware/yulin

npm:

https://www.npmjs.com/package/@kensio/yulin

Most JavaScript and TypeScript test frameworks provide assertions through APIs such as:

expect(user).toBeDefined();
expect(status).toBe("active");

These are readable and widely used, but they generally cannot communicate additional type information back to TypeScript. The assertion checks something at runtime, but the compiler is unable to follow the types through the fluent interface. This means that calling the assertion does not update the type information in the test.

Updating type information is what type narrowing means. The @kensio/smartass package does that via a TypeScript feature called assertion signatures.

This means that each assertion in a test provides more type information to TypeScript. This lets you avoid boilerplate type casting, as well as convert those type casts into runtime assertions so the test validation is stronger and clearer.

The basic idea

Consider a value that might be undefined:

const user: User | undefined = getUser();

A normal runtime check narrows the type:

if (!user) {
  throw new Error("User not found");
}

user.name;

TypeScript understands that user cannot be undefined after the check.

An assertion function allows us to package that inference of type information into a reusable function:

import { assertNonNullable } from "@kensio/smartass";

const user: User | undefined = getUser();

assertNonNullable(user);

user.name;

The assertion throws if the value is null or undefined, but it also tells TypeScript that user cannot be undefined. TypeScript uses that information to infer that user must therefore be an instance of the User type.

As a result, there is no need for optional chaining or non-null assertions afterwards. E.g. with vitest assertions:

const user: User | undefined = getUser();

expect(user?.name).toBe("foobar");

Due to the lack of type signatures in that fluent assertion chain, we have to use the null-chain operator ? for the rest of the test. In this example it’s trivial, but in more complex tests with larger object structures, it can lead to a lot of clutter in the test.

Moving the type inference into an explicit assertion function is also beneficial by leading to more detailed validation in the test:

import { assertNonNullable, assertIdentical } from "@kensio/smartass";

const user: User | undefined = getUser();

assertNonNullable(user);
assertIdentical(user.name, "foobar");

With this approach, if user is undefined, we get a clear assertion failure message. IDEs like WebStorm can also highlight the specific assertion that failed, and link to it from test output, which also speeds up debugging.

Why assertion signatures are useful

Many assertion libraries focus on runtime behaviour at the expense of the type system. They verify conditions and produce useful failure messages, but the compiler cannot usually use those assertions when analysing code. It seems like a shame to miss out on such a large potential benefit in TypeScript.

For example:

expect(user).toBeDefined();

user.name;

TypeScript still considers user to be User | undefined.

By contrast:

assertNonNullable(user);

user.name;

TypeScript knows user must be exactly User.

The assertion signature narrows the type, so IntelliSense, autocomplete and compile-time checking all benefit from the assertion.

This seems trivial with this tiny example, but other more advanced assertion functions in @kensio/smartass can provide precise type narrowing.

This becomes particularly useful when dealing with values coming from APIs, databases, configuration files or external systems where the runtime shape is often broader than the shape expected by the rest of the application.

Narrowing to specific values

Assertion signatures are not limited to null checking.

For example, assertOneOf() can narrow a value to a specific set of allowed values:

import { assertOneOf } from "@kensio/smartass";

const status = getStatus();

assertOneOf(status, [
  "pending",
  "active",
  "completed",
]);

// status is now:
// "pending" | "active" | "completed"

The assertOneOf() function doesn’t just tell TypeScript that the input is of type string. It narrows the type down as much as possible to the literal union type.

Array assertions

The package also includes assertions that provide stronger guarantees about arrays.

For example:

import { assertArrayNotEmpty } from "@kensio/smartass";

const users: User[] = getUsers();

assertArrayNotEmpty(users);

const firstUser = users[0];

After the assertion, TypeScript understands that the array contains at least one item, so there’s no need for the null-chaining operator ? on users[0].

You can also assert on specific numbers of items in an array with similar precise type narrowing:

import { assertArrayLength } from "@kensio/smartass";

const users: User[] = getUsers();

assertArrayLength(users, 3);

const thirdUser = users[2];

Again, the assertArrayLength() function is able to tell TypeScript that users[2] exists. TypeScript can then infer from there that users[2] must be of type User.

Runtime validation and type information together

One of the recurring themes in TypeScript codebases is that runtime validation and static typing are often treated as separate concerns. Runtime validation is used in tests to confirm expected behaviour, while the type system might be used to provide some haphazard assistance and linting.

Assertion functions with type narrowing assertion signatures bring those two concerns together in tests.

Instead of validating a value and then manually convincing TypeScript that the value is safe, the assertion function lets you get both benefits together in one line.

The primary goal is not to create another testing DSL but to provide small assertion functions that improve both runtime safety and type inference. The assertion functions can be used in any test framework such as Jest or Vitest.

At the time of writing, `` provides the following assertion functions, each of which has the most precise type narrowing possible:

• assertArrayEquals
• assertArrayIncludes
• assertArrayIncludesAll
• assertArrayLength
• assertArrayMinLength
• assertArrayNotEmpty
• assertBufferEqual
• assertFalse
• assertIdentical
• assertInstanceOf
• assertNonNullable
• assertNumberBetween
• assertNumberToNearest
• assertObjectEquals
• assertObjectMatches
• assertOneOf
• assertStringEndsWith
• assertStringIncludes
• assertStringNotIncludes
• assertStringStartsWith
• assertThrowsError
• assertThrowsErrorAsync
• assertTrue
• assertTypeBigInt
• assertTypeBoolean
• assertTypeFunction
• assertTypeNumber
• assertTypeNumeric
• assertTypeObject
• assertTypeString
• assertTypeTypedArray
• assertUndefined
• assertUuidV4

Installing it

npm: https://www.npmjs.com/package/@kensio/smartass

GitHub: https://github.com/KensioSoftware/smartass

Docs: https://smartassertions.dev/

When writing tests, it’s common to need objects that are mostly boilerplate. A test might only care that a user has a particular email address, but creating that user object may require a username, profile settings, timestamps and a collection of other properties. As those objects grow, the setup code can easily become more prominent than the behaviour being tested.

Part Factory can help to make that test setup more concise and readable. A factory defines a complete default object, and each test overrides only the properties that are relevant to the scenario being tested.

The basic idea

A factory contains the default values for an object:

interface Foo {
  name: string;
  size: number;
}
import { StaticFactory } from "@kensio/part-factory";

const fooFactory = new StaticFactory<Foo>({
  name: "Foobar",
  size: 10,
});

const defaultFoo = fooFactory.make();
// { name: "Foobar", size: 10 }

const myFoo = fooFactory.make({ size: 20 });
// { name: "Foobar", size: 20 }

Instead of repeating every property in every test, the test only specifies what is different. The defaults remain in one place, making the test setup easier to read and reducing the amount of noise surrounding the important assertions.

There is a recursive DeepPartial<T> type in @kensio/part-factory, so TypeScript can follow the partially overridden properties in each test case.

Why use a factory pattern?

For small objects with a couple of properties, plain object literals are unproblematic. The benefit of a factory pattern like this becomes more obvious when the same type appears throughout a test suite, when objects are deeply nested, or when application types evolve over time.

A common maintenance problem is that object construction becomes scattered across dozens or hundreds of tests. If a type gains a new required property, many tests may need updating even though they have no interest in that property. By centralising defaults inside a factory, those changes can be handled in one location.

Tests that care about the new property can still override it explicitly, while the rest continue to use the default value provided by the factory.

Static and dynamic defaults

Part Factory provides two approaches to default values.

StaticFactory uses fixed values, which is enough for simple test data where we don’t need to test differing values. AWS SDK event and request structures often work well with static factories.

The alternative DynamicFactory generates fresh values each time a test object is created, making it useful for IDs, timestamps, names and other generated data where we want to test differing values.

import { DynamicFactory } from "@kensio/part-factory";
import { faker } from "@faker-js/faker";

const fooFactory = new DynamicFactory<Foo>(() => ({
  name: faker.word.noun(),
  size: faker.number.int({ max: 100 }),
}));

const defaultFoo = fooFactory.make();
// { name: "external", size: 42 }

const myFoo = fooFactory.make({ size: 20 });
// { name: "front", size: 20 }

The values can come from any function, so libraries such as Faker work naturally, but there is no dependency on any particular data-generation tool.

Variant factories

Many test suites contain named variations of a common object. A system might have a normal user, an administrator, a suspended account or a user with incomplete profile data. These are usually not entirely different objects but are only modifications of a shared baseline.

VariantFactory provides that kind of variation on a baseline:

import { DynamicFactory, VariantFactory } from "@kensio/part-factory";
import { faker } from "@faker-js/faker";

const animalFactory = new DynamicFactory<Foo>(() => ({
  name: () => faker.animal.type(),
  size: () => faker.number.int({ max: 100 }),
}));

const zebraFactory = new VariantFactory<Foo>(animalFactory, {
name: "Zebra",
});

const zebra = zebraFactory.make();
// { name: "Zebra", size: 42 }

const largeZebra = zebraFactory.make({ size: 100 });
// { name: "Zebra", size: 100 }

This allows common variations to become named concepts within the test suite without duplicating the entire object definition. It also combines well in the situation described above where the baseline object structure changes, because the variant factories don’t need to be changed, only the shared baseline.

Nested overrides

Overrides are implemented as deep partials. If a test only needs to modify a single nested value, it does not need to replace the entire nested object.

For example, if a user contains a nested address object, a test can override just the city while keeping the rest of the default address intact. This is useful for API responses, configuration objects, CMS data structures and other deeply nested data models.

Strong typing

The package is TypeScript-first, so factories understand the shape of the objects they produce, allowing TypeScript to validate both default values and nested overrides.

Invalid property types are caught at compile time, and overrides cannot introduce properties that do not exist on the target type. This helps keep test helpers aligned with application types rather than becoming a loosely typed layer that gradually drifts away from the real data model.

What Part Factory is not

Part Factory is intentionally small in scope. It is not a fixture framework or fake data generator, and it does not deal with object persistence or test runners.

All it does is create plain objects with default values and strongly typed nested overrides. More advanced behaviour can always be built on top when required, but many unit and integration tests only need a convenient way to create representative objects.

Installing it

npm: https://www.npmjs.com/package/@kensio/part-factory

GitHub: https://github.com/KensioSoftware/part-factory

This certification focuses on knowledge of:

• AWS CodePipeline
• AWS CodeBuild
• AWS CodeDeploy
• AWS CloudFormation
• Amazon CloudWatch
• AWS Systems Manager
• AWS Identity and Access Management (IAM)
• AWS Lambda
• Amazon Elastic Container Service (ECS)
• AWS Organizations

https://cp.certmetrics.com/amazon/en/public/verify/credential/54415a7ade114a47b2c7fa656a669be1

AWS Certified DevOps Engineer Professional (DOP-C02) Certificate PDF

This certification focuses on knowledge of:

• Amazon VPC
• AWS IAM
• AWS Organizations
• AWS Direct Connect
• AWS Site-to-Site VPN
• Amazon Route 53
• Amazon EC2
• Elastic Load Balancing
• Amazon S3
• Amazon RDS / Aurora
• AWS CloudFormation

https://www.credly.com/earner/earned/badge/224c5eb1-956f-4b02-8170-72538488f8e0

This certification focuses on knowledge of:

• AWS Lambda
• Amazon DynamoDB
• AWS IAM
• Amazon API Gateway
• Amazon SQS
• Amazon S3
• Amazon CloudWatch
• Amazon SNS
• Amazon EventBridge
• AWS CodePipeline

https://www.credly.com/earner/earned/badge/4b2991ab-5568-4b2e-8a53-b73968c244b6

This includes handling the TLS certificate in AWS ACM. Unfortunately it seems it’s either difficult or impossible to do this in a single CloudFormation stack, because the ACM certificate must be created in the us-east-1 region, and there’s no straightforward way to then depend on and access that resource for resources being deployed in another region (eu-west-2 in my case).

Because of that, I’ve split it into two separate CloudFormation stacks, one that handles the Hosted Zone in Route53 and the ACM Certificate for it, which is deployed in us-east-1, and then another stack for everything else, which is deployed in eu-west-2.

The CloudFormation template for the Route 53 Hosted Zone and ACM Certificate looks like this:

AWSTemplateFormatVersion: "2010-09-09"
Description: Hosted Zone and ACM Certificate for static website.

# NOTE: If the domain name is not registered in Route 53, you'll need to begin
# the creation of this stack, wait for the Route53 Hosted Zone to be created,
# then go and take the name servers for it in Route 53 and set them in the
# registrar (e.g. Namecheap) so that DNS validation of the ACM Certificate can
# complete. It can take more than 30 minutes for the validation to happen after
# this.

Parameters:
  DomainName:
    Type: String
    Description: The domain name for the static site.

Resources:

  Route53HostedZone:
    Type: AWS::Route53::HostedZone
    Properties:
      Name: !Ref DomainName

  DomainCertificate:
    Type: AWS::CertificateManager::Certificate
    Properties:
      DomainName: !Ref DomainName
      ValidationMethod: DNS
      DomainValidationOptions:
        - DomainName: !Ref DomainName
          HostedZoneId: !Ref Route53HostedZone

Outputs:
  CertificateArn:
    Value: !Ref DomainCertificate
    Description: ARN of ACM Certificate created inside StackSet.
    Export:
      Name: !Sub ${AWS::StackName}-cert-arn

You can deploy that with your choice of domain name like this:

export DOMAIN_NAME='example.com' && \
export ZONE_CERT_STACK='foobar-zone-cert-stack-name' && \
aws cloudformation create-stack \
  --region 'us-east-1' \
  --stack-name "${ZONE_CERT_STACK}" \
  --template-body file://path/to/domain-certificate.cloudformation.yml \
  --parameters ParameterKey=DomainName,ParameterValue="${DOMAIN_NAME}" \
  && \
aws cloudformation --region 'us-east-1' wait stack-create-complete --stack-name "${ZONE_CERT_STACK}" && \
aws cloudformation --region 'us-east-1' describe-stacks --stack-name "${ZONE_CERT_STACK}"

Note that if the domain name is not registered in Route 53, you’ll need to begin the creation of this stack, wait for the Route53 Hosted Zone to be created, then go and take the name servers for it in Route 53 and set them in the registrar (e.g. Namecheap) so that DNS validation of the ACM Certificate can complete. It can take more than 30 minutes for the validation to happen after this, because it has to wait for DNS to update.

Once that stack has deployed, you can find the ARN of the new ACM Certificate like this:

aws acm list-certificates --region 'us-east-1' \
  --query 'CertificateSummaryList[].[CertificateArn,DomainName]' --output text \
  | grep "${DOMAIN_NAME}"

You can manually store that Certificate ARN in an environment variable, or do it automatically in one command:

export CERT_ARN=$( aws acm list-certificates --region 'us-east-1' \
  --query 'CertificateSummaryList[].[CertificateArn,DomainName]' --output text \
  | grep "${DOMAIN_NAME}" | cut -f1 )

Then we use the main CloudFormation template to set up all the other AWS infrastructure for the static site. The template looks like this:

AWSTemplateFormatVersion: "2010-09-09"
Description: Static website set up.

Parameters:
  DomainName:
    Type: String
    Description: The domain name for the static site.
  CertificateArn:
    Type: String
    Description: ARN of the ACM Certificate.

Mappings:
  Region2S3WebsiteSuffix:
    us-east-1:
      Suffix: .s3-website-us-east-1.amazonaws.com
    us-west-1:
      Suffix: .s3-website-us-west-1.amazonaws.com
    us-west-2:
      Suffix: .s3-website-us-west-2.amazonaws.com
    eu-west-1:
      Suffix: .s3-website-eu-west-1.amazonaws.com
    ap-northeast-1:
      Suffix: .s3-website-ap-northeast-1.amazonaws.com
    ap-northeast-2:
      Suffix: .s3-website-ap-northeast-2.amazonaws.com
    ap-southeast-1:
      Suffix: .s3-website-ap-southeast-1.amazonaws.com
    ap-southeast-2:
      Suffix: .s3-website-ap-southeast-2.amazonaws.com
    ap-south-1:
      Suffix: .s3-website-ap-south-1.amazonaws.com
    us-east-2:
      Suffix: .s3-website-us-east-2.amazonaws.com
    sa-east-1:
      Suffix: .s3-website-sa-east-1.amazonaws.com
    cn-north-1:
      Suffix: .s3-website.cn-north-1.amazonaws.com.cn
    eu-central-1:
      Suffix: .s3-website.eu-central-1.amazonaws.com
    eu-west-2:
      Suffix: .s3-website.eu-west-2.amazonaws.com

Resources:

  WebsiteDNSName:
    Type: AWS::Route53::RecordSet
    Properties:
      HostedZoneName: !Join [ '', [ !Ref DomainName, . ] ]
      Comment: CNAME redirect custom name to CloudFront distribution
      Name: !Ref DomainName
      Type: A
      AliasTarget:
      	# https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-route53-aliastarget.html
        HostedZoneId: Z2FDTNDATAQYW2
        DNSName: !GetAtt WebsiteCDN.DomainName

  S3BucketForWebsiteContent:
    Type: AWS::S3::Bucket
    Properties:
      BucketName: !Ref DomainName
      AccessControl: PublicRead
      WebsiteConfiguration:
        IndexDocument: index.html
        ErrorDocument: error.html

  WebsiteCDN:
    Type: AWS::CloudFront::Distribution
    Properties:
      DistributionConfig:
        Comment: !Ref DomainName
        Aliases:
          - !Ref DomainName
        Enabled: true
        PriceClass: PriceClass_100
        HttpVersion: http2
        DefaultCacheBehavior:
          ViewerProtocolPolicy: redirect-to-https
          # https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/using-managed-cache-policies.html
          CachePolicyId: 658327ea-f89d-4fab-a63d-7e88639e58f6
          TargetOriginId: only-origin
          Compress: true
        Origins:
          - CustomOriginConfig:
              OriginProtocolPolicy: http-only
            DomainName: !Join ['',[
              !Ref 'S3BucketForWebsiteContent',
              !FindInMap [Region2S3WebsiteSuffix,!Ref 'AWS::Region', Suffix]
            ]]
            Id: only-origin
        CNAMEs:
          - !Ref DomainName
        ViewerCertificate:
          AcmCertificateArn: !Ref CertificateArn
          MinimumProtocolVersion: TLSv1.2_2019
          SslSupportMethod: sni-only

  UpdateSiteUser:
    Type: AWS::IAM::User

  UpdateSiteUserAccessKey:
    Type: AWS::IAM::AccessKey
    Properties:
      UserName: !Ref UpdateSiteUser

  UpdateSiteUserAccessKeySecret:
    Type: AWS::SecretsManager::Secret
    Properties:
      Name: !Sub ${AWS::StackName}-update-site-user-access-key-secret
      Description: !Sub "Access key credentials for ${AWS::StackName} site update user."
      SecretString: !Sub
        - '{"AccessKeyId":"${AccessKeyId}","SecretAccessKey":"${SecretAccessKey}"}'
        - AccessKeyId: !Ref UpdateSiteUserAccessKey
          SecretAccessKey: !GetAtt UpdateSiteUserAccessKey.SecretAccessKey

  UpdateSitePolicy:
    Type: AWS::IAM::Policy
    Properties:
      PolicyName: !Sub update-${AWS::StackName}-site-policy
      PolicyDocument:
        Version: "2012-10-17"
        Statement:
          - Effect: Allow
            Action:
              - s3:PutObject
              - s3:PutBucketPolicy
              - s3:ListBucket
              - s3:DeleteObject
              - s3:PutObjectAcl
              - cloudfront:CreateInvalidation
              - s3:GetBucketPolicy
            Resource:
              - !Sub
                - "arn:aws:cloudfront::${AWS::AccountId}:distribution/${WebsiteCDN}"
                - WebsiteCDN: !Ref WebsiteCDN
              - !GetAtt S3BucketForWebsiteContent.Arn
              - !Join ["", [!GetAtt S3BucketForWebsiteContent.Arn, "/*"]]
      Users:
        - !Ref UpdateSiteUser

Outputs:

  BucketName:
    Description: Name of S3 bucket for website content.
    Value: !Ref S3BucketForWebsiteContent

  CloudfrontDistribution:
    Description: CloudFront distribution ID.
    Value: !Ref WebsiteCDN

Note that for the next step, the CERT_ARN environment variable needs to be set as described above. You can then deploy a stack from that template like this:

export DOMAIN_NAME='example.com' && \
export MAIN_STACK='foobar-stack-name' && \
aws cloudformation --region 'eu-west-2' create-stack \
  --stack-name "${MAIN_STACK}" \
  --template-body file://path/to/cloudformation.yml \
  --capabilities CAPABILITY_IAM \
  --parameters ParameterKey=DomainName,ParameterValue="${DOMAIN_NAME}" \
               ParameterKey=CertificateArn,ParameterValue="${CERT_ARN}" \
  && \
aws cloudformation --region 'eu-west-2' wait stack-create-complete --stack-name "${MAIN_STACK}" && \
aws cloudformation --region 'eu-west-2' describe-stacks --stack-name "${MAIN_STACK}"

That stack includes an IAM user with granular permissions for updating the static site, and an access key for that user. To get the access key id and secret access key for the site update user:

export MAIN_STACK='foobar-stack-name' && \
aws secretsmanager get-secret-value --secret-id "${MAIN_STACK}-update-site-user-access-key-secret"

Take those access key credentials and use them set two secrets in the GitHub repo for the project: AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY.

After that, add a GitHub Action workflow in the project at .github/workflows/update-site.yml. The content of the GitHub Action file is:

name: Update site

on: push

jobs:
  build:
    name: Build and Deploy
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v1
      - name: Install Hugo
        run: |
          HUGO_DOWNLOAD=hugo_extended_${HUGO_VERSION}_Linux-64bit.tar.gz
          wget https://github.com/gohugoio/hugo/releases/download/v${HUGO_VERSION}/${HUGO_DOWNLOAD}
          tar xvzf ${HUGO_DOWNLOAD} hugo
          mv hugo $HOME/hugo
        env:
          HUGO_VERSION: 0.94.2
      - name: Use Node.js
        uses: actions/setup-node@v1
        with:
          node-version: '16.x'
      - name: NPM install
        run: |
          npm install
          npm install -g postcss-cli autoprefixer
      - name: Hugo Build
        run: $HOME/hugo -v
      - name: Deploy to S3
        if: github.ref == 'refs/heads/main'
        run: aws s3 sync public/ s3://example.com/ --delete --acl=public-read && aws cloudfront create-invalidation --distribution-id=ABC123CFDISTID --paths=/*
        env:
          AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
          AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          AWS_EC2_METADATA_DISABLED: true

Note that you need to set the S3 bucket name and CloudFront Distribution ID in the GitHub Action file.

You should now have push-to-deploy with the static site content being generated in the Github Action and then pushed to S3.

If you need any help with AWS development work, then contact me.

I worked in permanent roles at various start ups in London for about eight years, and in 2021 I made the switch to contracting and freelancing. I knew that as a contractor I would be thrown into all sorts of projects with different levels of design, development and maintenance. One thing that has struck me from taking on this kind of work is that I actually feel a vindication of the “excessive” testing described above.

For example, in a permanent role in the past, I developed a system involving an XML message processor which needed to verify XML signatures on the incoming messages. At the time, I wrote tests that generated a fresh private key and certificate, signed an XML message with the key, configured the system under test to use the certificate, and confirmed that it could verify the signed XML and also perform whatever business logic was being covered in each particular test case. A lot of this might seem a bit excessive or unnecessary for testing a fairly bog standard system.

For example, it might have been easier to not bother with the XML signature and verification in tests, as that part was mostly just calls to a library anyway. Alternatively, it might have been easier to just generate a test private key and certificate manually and perhaps keep it in a password manager somewhere or even commit it into the repo as it’s only for testing purposes.

However, having the test code generate fresh keys and certificates to use on the fly does have some benefits. It exercises more of the system from start to end. The tests confirm the configuration of the system and its use of the library, as well as the business logic. If we need to upgrade the library or switch to another one, the tests will be able to validate that change with quite a lot of confidence. Having the key and certificate generation code committed in the repo is also a form of documentation of how that works, and makes it easier to modify the tests in future if we switch to a different signing algorithm or key strength. It also means the tests “just work”, with no set up required other than running the tests.

With more detailed and extensive tests, it’s less mysterious what is happening in the system and in the tests, and that can be useful for people working on both of those in the future. It can save money for the company, because future development can happen quickly, confidently and safely by leaning on these kinds of tests.

Coming back to the contracting work, the above was somewhat vindicated recently when I worked on a similar system involving XML messages with signatures. In this case, whoever built the system in the past hadn’t included tests that covered the signature verification part of the system. Lo and behold, by the time I came in to work on this project, the changes that were needed involved modifying the signature verification as well as how the XML structure was parsed. It took a lot of digging around in sparse documentation, trawling through old commits and tracking down people who might have some more knowledge. This was the only way to gather all of the necessary information to make the changes safely and then deploy them with confidence. A lot of the time and effort could have been saved had more extensive tests been put in at the time of development.

In summary, I’d say this confirms that it’s always worth considering other people or your future self when deciding how detailed and extensive to make your test cases. More often than not, it will pay off to cover more of the system with explicit tests.

I’m always keen to help early stage start ups with building an MVP, so contact me and we can discuss working together.

If you need an experienced contract software engineer in the UK on a remote basis, please get in touch.

There are indeed many such tradeoffs. Serverless approaches offer many benefits, in particular the potential for cost savings (at least in the early stages of a project’s lifecycle) and the chance to offload what would otherwise be a lot of operational management work on to the cloud provider, letting you focus on your building your business and not an abstraction castle.

Potential for greater complexity

This isn’t even an exaggerration of the complexity of most serverless and cloud architectures I have worked with. Cloud providers like AWS and GCP make it very easy to add ever more nodes to the architecture diagram, as they are more than happy to charge you for all of these managed services. This needs to be balanced against the potential for cost-savings by handling the fundamentals yourself, and of course the eternal benefits of keeping things simple.

Lack of observability / lack of ability to audit

Because serverless approaches can make it easy to continually add more and more individually deployed functions, there can be a combinatorial explosion of potential execution paths through the distributed system as the functions interact with each other in myriad ways. This can make it difficult to observe what is happening at the logical level, or to track down bugs hiding amongst obscure edges in the execution graph.

This point might be more debatable than the rest, as there is an argument that you get better observability with serverless architectures because you can easily separate out compute, storage, running costs and so on down to the function level.

Database connection count

Pretty much every company that uses serverless is going to hit the max database connections issue. With every function invocation in its own container demanding its own database connection, its easy to exceed the maximum number of concurrent connections that the database can handle.

Having said that, this was already a problem with elastic autoscaling servers in the cloud – it’s perfectly possible to scale up enough servers to exceed your database’s max connections limit.

There are various solutions to this, including database connection pooling, CQRS, read-replicas and so on.

Vendor lock-in

Another common objection to serverless approaches is the high potential for vendor lock in, as you’re tied into the serverless technology of whichever cloud platform you use. Cloud providers tend to encourage this by pushing features such as vendor-specific function triggers that tend to rely on use of their SDK to be convenient.

If you treat serverless as more of a deployment strategy than an architectural paradigm, you might be able to mitigate this issue to some extent by depending less on vendor-specific offerings, or at least depending on them less directly.

You can see the search system at Moment Wall Art, for example by searching for “Japanese river”.

The e-commerce sales system is primarily a static website to ensure high scalability and fast response times, with low operating costs. The new search solution builds on top of this static system by pre-building indexes of the product data into individual JSON files that are separated by keyword. These keyword-based JSON index files are deployed alongside the rest of the e-commerce website in AWS S3 with CloudFront in front of them as a distributed caching layer.

When a customer searches for art pieces to buy on Moment Wall Art, a small piece of vanilla JavaScript code fetches the relevant JSON index file for each keyword, each of which contains optimised product data relevant to their respective keyword.

The product data in the keyword index files is then aggregated and rendered to the page for the customer to browse through.

Due to the front end optimisation and static back end powering this solution, search result pages are highly responsive, and the solution has low operating costs.

Contact me about custom search solution development.

The Etsy synchronisation system is implemented in Python. It reads and writes between Moment Wall Art’s bespoke e-commerce system and the Etsy API. Product data including descriptions, images and pricing is kept up to date in Etsy, while orders are synchronised back from Etsy into the e-commerce platform for easier order processing and inventory management.

This integration runs in AWS Lambda as part of the serverless architecture behind Moment Wall Art. Using serverless functions to run ad-hoc workloads lowers costs and saves time that would otherwise need to be spent managing servers and other infrastructure pieces.

Contact me about Etsy API integrations and development.

The new WordPress theme is more efficient both on the backend when serving user requests, and in its frontend CSS and JavaScript, compared to many off-the-shelf options.

The new design is also mobile-first and responsive across a range of different devices and network connectivity situations. This allows a greater number of readers to benefit from Hacking Chinese’s educational content.

Contact me about WordPress software development.

The customised Etsy integration syncs product catalogue data from Pop Robin Cards’ e-commerce system into the Etsy shop. This includes product names and images, as well as Etsy-specific product descriptions that are generated from product data using templates.

The integration is also able to make sure that other useful information is included in the Etsy listing for each product, including shop notices and how to add personalised gift messages when buying via Etsy.

Product inventory is also handled automatically, with stock levels being updated from the main e-commerce system into Etsy, and also decremented in the main system when orders are received in Etsy. This has reduced instances of out-of-stock products and improved the customer experience for customers on both platforms.

Orders are regularly synchronised back from Etsy into the central e-commerce platform to provide a single order view for packing and dispatching orders. This has improved efficiency and increased accuracy in order dispatch.

As all sales data is synced back into the main system, Pop Robin Cards is able to benefit from aggregated statistics covering both sales platforms.

Contact me about development of software integrations.

The e-commerce system manages Moment Wall Art’s product catalogue, including a detailed taxonomy system. This allows for specific catalogue pages such as Japanese wall art and Alphonse Mucha Prints, letting customers find art pieces that interest them.

Despite being serverless with a static web frontend, the e-commerce website has a full product search system. This allows searching for art prints using keywords such as “Mucha nouveau” to find specific art pieces that a customer might be interested in. The search solution uses pre-generated JSON index files that are included in the static site build and deployed to S3 for high-speed responses at low cost.

Stripe is integrated as the payment handler, providing a streamlined checkout experience and secure transactions. This also allows the checkout process to remain serverless, keeping operational costs low while maintaining high availability.

The e-commerce system is also integrated with Etsy via their API, with inventory synchronized out and orders synchronized back in to the main system.

Contact me about e-commerce software development.

The website is a serverless solution with a static web frontend. This ensures high availability with low running costs, as well as fast loading times for users.

The web development project includes a Chinese example sentence search tool for users to find example usages of Chinese vocabulary that they are studying. This is powered by a Postgres SQL database and an ElasticSearch fulltext search engine.

Another software tool that was implemented is a pinyin conversion tool, which allows users to generate pinyin romanisation from Chinese text. This is implemented in PHP within the Laravel framework.

Contact me about web development.

Pop Robin Cards sells 3D pop up cards such as birthday cards and Christmas cards. The bespoke e-commerce solution that I developed manages the inventory and sales process end-to-end, including stock management, search, payment and customer communications via e-mail.

The e-commerce product search system allows search for greetings cards, for example those most relevant to “cherry blossom”. This is powered by ElasticSearch with a Redis cache to keep it fast and responsive.

Multiple payment systems including PayPal and Stripe are integrated to give customers a choice of payment options. An Etsy API integration also syncs products and orders with Etsy to generate more sales on an alternative marketplace with centralised e-commerce management.

The web frontend is SEO optimised in several ways, including multiple taxonomy systems for internal linking and discoverability. Products are linked from an Occasion taxonomy, for example Valentine’s Cards. A Tag taxonomy indexes the themes and content of different product, such as Flower Cards. There is also a Suitable Recipient taxonomy that allows viewing products in groupings including “Cards for Father”. These taxonomies are also combined to make more specific and helpful categorisations such as “Birthday Cards for Father”.

This customised e-commerce platform also has several social media integrations. Every page has meta tags optimised for social media, so that the best title, description and social media image are selected when a page is shared on social media or messaging platforms. The system also makes automated posts to social networks based on trending products.

The order process includes a series of emails to inform the customer of the status of their order, from order placement, dispatch and tracking codes from Royal Mail.

Contact me about custom e-commerce development.

Due to a combination of high-quality content and good SEO, the education website receives a relatively high level of traffic, which was becoming expensive to host for WordPress. Ensuring that the WordPress installation was up-to-date and secure also required ongoing work. Converting the website from WordPress to a static website system made it cheaper to operate, faster to use and more secure.

A large part of the conversion work was extracting existing content from the WordPress MySQL database and generating Markdown files to replace it. A Python script automated the content conversion.

The WordPress theme was also replaced with a redesigned static theme with more efficient CSS to render more quickly in users’ browsers.

The new static website is hosted in AWS using S3, Cloudfront and Route53.

Contact me about WordPress conversion and AWS development.

Entity modelling, including entity relations.

Aggregate

Group of entities that can be interacted with as a single object.

Commands

Decoupled command objects that perform change operations on entities and aggregates.

Queries

Decoupled query objects that fetch single entities and collections of entities.

Validation

• Command, query, entity and aggregate validation.
• Static validation that can be defined at built-time.
• Dynamic validation that can interact with live state at run-time.
• Integrated with IAM so that different roles are permitted to perform different commands and queries.

Transitions / entity lifecycles

Ability to move entities through stages in a lifecycle. For example, draft products that can be published as products, or baskets that become orders via a checkout process. This makes validation more flexible by allowing a gradual assembly of a valid entity via intermediate stages.

Interfaces

Decoupled interfaces that allow interaction with the application. Any command or query can be issued via any supported interface.

HTTP

HTTP APIs, such as REST, OpenAPI, GraphQL.

Local CLI

Heavy command line interface that runs application code locally to issue commands and queries.

Network CLI

Light command line interface that issues commands and queries over the network to an API.

Web

Web user interface that issues commands and queries over the network to an API. Could be purely HTML based, or use a web framework such as React or htmx.

SDK

Abstract interface in a programming language for use by other developers to conveniently issue commands and queries over the network to an API.

Mobile / iOS / Android

Mobile application user interfaces that issue commands and queries over the network to an API.

Desktop

Desktop application user interface that issues commands and queries over the network to an API.

Slack

Slack chat integration built using the Slack SDK to allow issuing commands and queries via intermediate event handlers on the application side.

Events

• Entity events (create, update, delete)
• Configurable as synchronous or asynchronous via jobs.

Tasks

• Asynchronous jobs that are handled on a queue.
• Can bundle sub-tasks into a parent task and fan-out the handling.
• Can mark a task as dependent on another so that the dependent tasks are handled sequentially.
• Incremental progress can be reported back to a Job entity, which can then be pushed out via subscriptions or notifications.

Authentication

• User and service authentication.
• Third-party authentication via SAML.
• External authentication via event handlers such as Slack event handlers, or OS users for the local CLI.

Authorization / Permissions / IAM

• Static IAM that can be defined at build-time and hard-coded in the application. This is fast.
• Dynamic IAM that can be configured and applied at run-time, allowing it to fetch live state. This is slow.

Notifications

• Email
• SMS
• Outbound web-hooks
• Web push
• Mobile push
• Slack
• Telegram

Inbound web-hooks

Convenience for setting up inbound web-hook endpoints for other systems to call into and trigger events and audit trails.

Subscriptions

Allow clients to subscribe to updates via websocket.

Entity subscription

Subscribe to changes to a specific individual entity or aggregate.

Series subscription

Receive incremental updates to a series, allowing chat applications and live update feeds.

Scheduling

• Cron-like scheduling of any command or task on a recurring basis.
• One-off scheduling of a command or task at a specific time.

Batch processing

• Handle changes across many entities by batching up work and handling it in tasks.
• Allow spreading batch work over intervals, e.g. “hourly once a week”. This means that one batch is processed every hour, with all entities covered once every week.

Filtered search

• AKA faceted search or layered search.
• Build a search index of an entity or aggregate.
• Allow client to specify filters on configurable facets.
• Results are paginated.
• Filter metadata is returned with paginated results, showing the count of entities matching each facet value.
• Common in e-commerce applications.

Infrastructure

• Framework assists with infrastructure management, based on configuration of other resources such as entities, drivers and so on.
• Docker
• AWS CloudFormation
• Terraform

Drivers

• Resources as configurable providers so that the business logic is abstracted from implementation.
• E.g. entities and aggregates can be backed by a datastore driver, which can be provided as different SQL databases, DynamoDB, filesystem, Redis etc.
• Static drivers are configured at build-time, so the resulting application only has that static driver available to it.
• Dynamic drivers are configured at run-time, so that the same application build can switch between drivers, for example in different environments.

Deployments

• Support for deployment management by making deployments a first-class concept and assisting with configuration of deployment tools such as Github Actions, CodeBuild and so on.

Logging

• Default logging on entities, aggregates, commands, queries, events, tasks.
• Driver-backed to allow configuring different log drivers.
• Integrates with infrastructure to set up logging appropriate to configured infrastructure.

Monitoring, alerting

• Driver-backed to allow configuring different log drivers.
• Integrates with infrastructure to set up monitoring appropriate to configured infrastructure.

Migrations

• Datastore migrations.

Auditing

• Support audit-trail logging to record commands, queries and tasks.

Introspection

• Application can dynamically inspect itself at runtime and show infrastructure data such as queue length.

Tests

Assistance with application test at different levels.

Unit

Tests for self-contained individual units such as classes and functions that don’t have any dependencies outside of the constructor or function arguments.

Process

Tests that run in a single OS process, with any external dependency either replaced with an in-process driver or mocked out with tools such as httpx-mock and moto. These are highly effective for debugging business logic, but will miss issues in the integration between real resources.

Integration

Tests that run entirely on the local OS, with services like databases running in Docker containers, and third-party APIs either mocked out with tools such as httpx-mock or as dummy implementations running in Docker containers. The application under test runs in the same process as the tests to allow debugging during integration tests.

Deployed / smoke

Tests that run against the application in deployed infrastructure such as AWS. These are few in number and aim for high-level sanity checks or smoke tests.

Environments

Runtime environment awareness which lets the application behave differently in different environments. Drivers can be specified per environment, for example the local development environment might use a different queue driver than the production deployment environment.

Local development

Feature flags

Configuration / settings

• Static configuration is specified at build-time and supplied to the application via environment variables or secure secrets. It cannot be changed at run-time. It only needs to be loaded once at application start-up.
• Dynamic configuration can be changed at run-time, either by administrators or users. It needs to be loaded before each relevant action in case it has changed.

Demos

Assistance for demoing features, for example a prompted command-line script that takes the demo presenter through a particular use-case.

Documentation

Assistance for documenting the application, for example by producing OpenAPI specifications.

Third-party interfaces

Support for modelling third-party APIs, which can facilitate test implementations and libraries.

Export

Ability to bulk export entities and aggregates in formats such as CSV and JSON.

Import

Ability to bulk import entities and aggregates from formats such as CSV and JSON. Can use tasks that handle the import asynchronously and report granular issues with individual rows.

Synchronisation

Two-way synchronisation with another system, so that updates flow in both directions. For example, stock counts can be updated both to and from a third-party sales system such as Amazon or Etsy.