Skip to main content

· 8 min read

Key Updates

Expo Mobile App Updates

With recent changes to both expo and the release of iOS16 we have made the usual annual changes to catch up with the latest changes.

  • Expo SDK46+ changed the way that OTA updates are published and we have updated the build process to support this in hamlet
  • Support has been updated around fastlane and the use of the Transporter app for mobile updates to TestFlight
  • All configuration for signing, publishing and controlling the mobile apps has been moved out of settings into solution based configuration. This makes the configuration available from our docs site and added defaults and validation to make sure it works as expected

bash

  • (expo): update cli usage for SDK 46+
  • (expo): testing updates for expo publish (#350)
  • workaround for fastlane missing support for transporter move
  • make search for signing more specific
  • include symbols disable and bitcode
  • disable signing on 3rd party libraries
  • ensure manual signing enabled on pod project
  • (expo): sdk differences in updates
  • (expo): use right app version when picking cli
  • (expo): align publish script with engine
  • (expo): reduce command line options
  • deprecate turtle and move to prebuild process
  • (expo): reinstate archive upload for sourcemaps (#349)

engine

  • (mobileapp): remove default on expo id override (#2045)

aws

  • (mobileapp): lookup for firebase properties (#641)
  • (mobileapp): testing updates

python

  • remove deprecated expo_app_publish param

AWS S3 Public Access Block By default

Inline with best practice security and those pesky news reports we have now enabled public access blocking across S3 buckets by default.

  • ACL based public access for objects can not be enabled by default within the hamlet aws implementation
  • To use policy based public access you can use the S3 component public access which will allow for bucket policy updates
  • The App Public permission has been removed which created an automatic public share on the ops data bucket. To ensure limited exposure we recommend creating an explicit S3 bucket for public access

engine

  • (s3): block public access on account s3

aws

  • (s3): disable public access by default
  • remove app public permissions attribute
  • remove app public attributes

Data Catalog Component

This new component represents services like AWS Glue which are used to provide a catalog of data sources that used across a system along with a centralised schema reference point. The focus of the initial release was to add support for creating a data catalog of AWS Service logs which are stored in S3.

Details on the module are available in PR #647) but essentially if you need to access any AWS services used in hamlet that log to S3 ( cdn/CloudFront, lb/Application Load Balancer) you can add the module and get the logs added into Glue and can make queries of the data using Athena

core

  • (datacatalog): add new component (#2049)
  • (datacatalog): initial support using Glue

aws

  • (module): add aws s3 service log datacatalog (#647)
  • (catalog): add base testing
  • SerDe naming

WAF Rule Engines

Web Application Firewalls offered by Cloud Providers have been extended to support more functionality including managed rules, rate limiting and more complex matching capabilities. In this release we have extended the support in hamlet to support these different WAF rules with the idea of a Rule engine. The engine is how a given rule is processed.

The engines included in this release are:

  • Rate Based: IP based tracking of requests and the enforcement of a rate limit for a given client IP
  • Vendor Managed: Rules that can be added to your WAF that are managed by a third party vendor. These rules are generally updated to align with the latest security incidents or over collective knowledge and automated processes developed by the vendor

core

  • (waf): add rule engines
  • (wafrule): engine type spelling (#2058)
  • (waf): cleanup of shared provider setup

aws

  • (waf): add support for more wafv2 rules
  • (waf): various fixes in WAF Handling

Extended CDN Support for Complex Scenarios

The CDN component has been updated to support more complex scenarios when dealing with distributions that perform complex routing or have shared caching policies across different parts of a site

This included the addition of some new subcomponents for the CDN:

  • origin is a new subcomponent which represents a single origin that can be consumed by multiple routes
  • response policy adds support for CDN providers which can modify things like response headers as they are returned to a client. For example, AWS CloudFront can now inject common security headers like HSTS or CORS if the origin doesn't include them
  • cache policy CDN Cache Policy defines a caching policy which can be used across multiple routes to standardise how caching is performed

You don't need to use these additional components if you don't want to and all existing support for controlling these options via the route are still in place.

core

  • (cdn): enable/disable error response overrides
  • (cdn): extended support for managing CDNS
  • (cdn): Response Policy Child reference

aws

  • (cdn): origin request policy headers (#657)
  • add default header policy for placeholders
  • (cdn): add enable/disable for error responses
  • (cloudfront): complex cdn scenarios
  • (cdn): use cloudformation to find cdn id
  • (cdn): add type checks and fix resource name
  • (cdn): add testing and fixes
  • (cdn): Redirect processing (#643)

Dynamic Values

IP Address Group resolution to CIDR ranges has been added as a dynamic value source, while we support IPAdressGroups across components that use IP Access control. The dynamic value support lets you provide network details to container env vars for example.

Now that we have dynamic values it is easier to add composite values which use a dynamic value and some fixed content. With that in mind we've added callback and logout url solution level configuration to the userpool

core

  • (dynamicvalue): add IP Address group support
  • (userpool): add support callback urls solution

aws

  • (userpool): add solution callback urls

Logging Enabled by Default

For all components the default behaviour is now to enable logging if we can. While this could add some additional costs when deploying to cloud providers, the cost of regret when you forget to enable logging and need it in an incident is much higher. So we turn it on now to help support your deployments.

core

  • enable logging by default
  • (network): add prefix control for flow logs

aws

  • (network): add control over flowlog prefix (#648)

Overall Hamlet changes

Below are the minor changes and fixes made across hamlet overall.

Bash Executor

Full set of changes: 8.7.0...latest

New Features

  • support cliv2 kms encrypt (#345)

Fixes

  • add an empty suffix for running on MacOS (#353)
  • update sourceMappingURL value after bundle rename
  • adjust sentry url prefix react-native apps (#351)

Engine

Full set of changes: 8.7.0...latest

New Features

  • (globaldb): add enable support for indexes
  • migrate tiers and solution setup into configuration data approach
  • district based solutions
  • add support for base attribute set expansion
  • add support for links as image sources
  • extend missing link detail error message

Fixes

  • (dynamicvalue): correct the output for tasks (#2057)
  • set tier indexes for tiers not in network
  • tier network configuration (#2046)
  • (dynamicvalues): support objects in replace (#2043)
  • nested expansion of attributes (#2040)
  • split expand base configuration (#2039)
  • set fixed index for shared tier
  • index handling for network ordering
  • tier lookup id name
  • configuration loading
  • update component settings for dynamic values

Refactorings

  • dynamic value handling (#2042)
  • add apigw certificate
  • move AV configuration to shared

aws

Full set of changes: 8.7.0...latest

New Features

  • (cw): ensure lambda is deleted with canary
  • (globaldb): add support for enabling indexes

Fixes

  • engine case for rule lookup
  • (network): handle missing subnet lookup
  • (s3): oai permissions (#653)
  • (windows): logContent setup for windows logging
  • (ecs): handle container tagging for ecs version
  • dynamic value setup for aws secrets
  • smtp user permissions in module
  • (filetransfer): log group name for subscription
  • fall through on missing network
  • remove redundant line from script
  • (image): use container repository for images
  • (ecs): skip lb processing when no lb port
  • (efs): correct tag format for access points
  • (cloudwatch): dependencies on subscription
  • (dnszone): add deployment subset check
  • typo in test module
  • (spa): cdn reference for path
  • (cw): update permissions for cw logs to kinesis (#634)
  • (lambda): check deployment units on function

Refactorings

  • (network): checks for networked tiers (#645)
  • (mobileapp): build configuration updates
  • remove public app data prefixes
  • (network): remove use of segmentObject
  • (ec2): av migration to shared provider

Azure

Full set of changes: 8.7.0...latest

New Features

  • (network): manage vpn cipher configuration

Refactorings

  • use base level attribute sets (#307)

Core Engine

Full set of changes: [2.0.0, 8.6.0...latest](https://github.com/hamlet-io/engine-core/compare/2.0.0, 8.6.0...master)

Fixes

  • Dependency updates and updates to the bundled JRE version used

Python

Full set of changes: 9.23.0...master

Fixes

  • remove use of transport from httpx client

· 5 min read

This has been a pretty quiet release this round, a few helpful updates with bug fixes taking the front of stage. There is also some initial work in our image management process which will be progressed in our next release and includes using links for image sharing along with a more engine based approach to managing images.

Key Features

Extensions for more components

We've found that as solutions grow the need for minor tweaks on components are required. So we have added extensions to more components to allow for updates to resource policies and even the ability to add additional resources to a common component.

  • Core - (baseline): add extension support for data (#2028)
  • Core - add support for topic and queue extensions
  • AWS - (baseline): extension and policy support
  • AWS - queue topic permission extensions (#618)

Dynamic Value Evaluation

When runbooks were introduced they needed the ability to pass value references between steps. The solution to this was to add support for a format for referencing a dynamic value as part of another string. With this release we have extended this support to all components and introduced the ability to add different dynamic look up sources.

The syntax for a dynamic value is __<type>:<attribute>:<attribute>__ where type is the dynamic value provider type and a value provider provides attributes that are used in the lookup process.

Dynamic values can be used on any component string properties defined in the solution. They can be combined with other values in the same string as well. So you can have a url formatted like "https://__attribute:cdn:FQDN__" and when the dynamic value is replaced it will replace everything between __ ( included ) with the value that was looked up.

The initial release supports the following dynamic providers

  • attribute: look up an attribute of linked component
  • setting: use a setting of the component in its env var format
  • aws_secret: (AWS only) use CloudFormation Secrets manager integration to provide a secret value to CloudFormation

Adding more Enable controls across attributes

With support for qualification across the whole solution having the ability to define if something is enabled or not in solution configuration makes it easy to use qualification to control if that setting should be used in each environment. This release has seen the enabled option added to more component configuration options.

Component Updates

Lambda

Added support for defining aliases as fixed name references to a given version of a lambda function

Correspondent

Added support for configuring the channels used by correspondent to communicate with users

LB

AWS - Added support for using a lambda function as a backend to an application load balancer

Full Change Log

Core

New Features

  • (baseline): add extension support for data (#2028)
  • (port): add enable for port healthcheck (#2030)
  • (entrance): stackoutput entrance
  • add support for topic and queue extensions
  • tasks and extensions for images in runbooks
  • (image): add support for the image component
  • (lambda): lambda alias (#2015)
  • add support for dynamic value evaulation
  • (correspondent): add support for channel configuration
  • (lb): add enable support for conditions

Fixes

  • (dn): Test for missing domain name (#2008)
  • (dynamicinput): error message text (#2027)
  • set buildblueprint output file name (#2026)
  • schema entrance output filename (#2025)
  • typo for deployment test
  • (output): output prefix handling
  • typo in dynamicvalue return
  • (dynamicvalue): remove fatal on missing link
  • update naming for source
  • correct private key property name (#2017)
  • (dynamicvalue): fix typo in configuration scope
  • (dynamicvalue): resolve handling for lists (#2013)
  • comment update
  • handle empty values or object references

Refactorings

  • (runbook): support dynamic values in conditions
  • remove occurrence dependency and apply at the occurrence level

Others

  • (lambda): update lambda runtime versions (#2031)

AWS

New Features

  • (baseline): extension and policy support
  • (lb): lambda backend support for lb (#626)
  • queue topic permission extensions (#618)
  • (image): aws support for the image component
  • (runbook): add push_image runbook for images
  • adds extensions and tasks for image management
  • (lambda): lambda aliases (#606)
  • (dyanmicvalues): add support for aws secrets as dynamic values
  • (correspondent): add support for AWS pinpoint channels
  • (lb): add support for enabling/disbaling conditions

Fixes

  • incorporate feedback
  • (cert): raise error on invalid FQDN
  • support dns zone creation without network (#619)
  • (lb): expand permissions for lambda invoke (#629)
  • error messages for port lookups (#628)
  • (lb): protocol checks for nlb
  • (lb): action lookup for network load balancer (#623)
  • (computetask): windows directory creation (#622)
  • (directory): handle missing config connector
  • (healthcheck): add iam service to setup
  • (lb): create alerts across all occurrences
  • (waf): support v1 -> v2 migrations
  • (db): aurora cluster backups (#616)
  • (lambda): size checking (#609)
  • (apigateway): handle open ip address groups (#608)
  • (apigateway): reference versioned lambdas
  • (lb): standard error for invalid port mapping

Refactorings

  • (ecs): lg pass handling (#607)

Azure

Fixes

  • (lb): standard error for invalid port mapping

Bash

New Features

  • (expo): add support for app display name on IOS apps (#340)
  • add stackoutput entrance support
  • (expo): add support for firebase and plugins

Fixes

  • conventional commit support (#342)
  • incorporate feedback
  • incorporate feedback
  • (cli): disable paging for aws cli v2
  • (expo): include project details for plist (#341)

Refactorings

  • set credentials (#343)

· 10 min read

This release has been a pretty standard update with new components, tasks and general improvements, and bug fixes.

Nothing too drastic... but we have started putting together a list of larger changes that will be introduced in our next release.

Key Features

User Defined Tagging

You can now control the tags applied to your resources that have been deployed by hamlet. Tags have been broken up into two categories, Common and Additional.

  • Common tags are based on details of the occurrence that is being deployed. This includes the tags that we already provide in hamlet, and include things like environment, product, build reference and the generated name of the component
  • Additional tags are user defined tags. These work as a collection of key value pairs that you can apply as you need to. Using Deployment Profiles you can easily create a tagging policy across all of your deployments.

To use tags all components now have a Tags attribute

{
"Tags": {
"Additional": {},
"Common": {
"Component": "boolean",
"CostCentre": "boolean",
"Deployment": "boolean",
"Layers": "boolean",
"Name": "string",
"Prefix": "string",
"Solution": "boolean"
}
}
}

The additional tags are defined as objects underneath the Additional attribute

```json
{
"MyKey" : {
"Value" : "MyValue"
}
}

Tag configuration has been added to all components, some providers don't support tags for particular resources so they might not be included on all resources.

  • tagging control for occurrences

Support for Additional Values

Component attributes now support setting additional values on top of the default values they offer. This is handy when values are passed from hamlet components directly to providers. If the provider adds support for a new value, for example a new AWS lambda runtime, you can use it straight away without waiting for us to add the new runtime into hamlet. This is currently enabled on the lambda component and will be added to other components as we identify use cases for this configuration option.

  • (lambda): chg constraint on runtime to warning
  • (lambda): remove constraint on runtime

New Components

A few new components made it into this release.

  • Load balancer backends This is a new subcomponent on load balancers that defines a group of backend servers independent of their Port Mappings. This is useful when you have a set of conditional rules that all use the same backend servers. Now you can define one backend and link the Port Mapping to the Backend.
  • Log Store Is an instance of log storage standalone from a component. You could use this when you have a lambda function that writes application specific logs and you want to maintain separation between the logs generated by the lambda function and the application logs.
  • Data Stream Similar to the data feed component which looks after forwarding data to a destination, the data stream handles collecting large amounts of data and queuing it for processing by clients to consume.
  • DocDB Adds support for deploying Document based NoSQL hosts. It's similar to the db component which provides the same support for SQL based database hosts
  • DNS Zone Public or private (network attached) dns zones can be defined through hamlet and uses the standard domain based configuration to assign the zone name.
  • Certificate Authority Used for building private PKI infrastructure, the certificate authority component represents a single authority. Through links between certificate authorities you can define an authority chain in line with best practices for PKI.

These components were implemented in the following changes:

  • (lb): define backends independent from port mappings
  • (logstore): add engine parameter support
  • (logstore): add dedicated log storage component
  • (datastream): define data stream component
  • add docdb support (#1934)
  • (dnszone): add support for network based configuration
  • add certificate authority component
  • aws logstore implementation (#573)
  • (datastream): aws implementation of data stream (#575)
  • add docdb support (#521)
  • (dnszone): add support for private vpc zones
  • (certificateauthority): intial support with ACMPCA

New Tasks

Runbooks are becoming more and more useful for running operational tasks on your deployments. In this release the aws provider has been updated with more tasks;

  • SES Password Generation This generates a sig4 signature of an AWS_SECRET_ACCESS_KEY that can be used to send emails through SES using the SMTP based API

  • S3 Empty Bucket Removes all objects and their versions from an S3 bucket

  • S3 Download Bucket Copies the contents of an S3 bucket to a local directory

  • Secrets Manager Get Secret Gets the value of a secret stored in AWS Secrets manager

  • add SES SMTP user module

  • (task): add ses smtp password generation task

  • add s3 runbook tasks

  • (task): add secrets manager get secret task

Policy Chunking for AWS

hamlet generates fine grained IAM permissions policies for each resource it deploys. This is great from a security best practice approach, however AWS does have limits on the size of an IAM policy. Some of our users were having issues with the generated policy hitting the policy size limits with complex deployments. To support this when generating IAM policy documents we split the permissions across multiple policies automatically.

  • policy chunking (#545)

Build Details

When working with images for application deployments its useful to know which version of an image is being used in a given environment. You can now find this quickly using the cli.

Running hamlet release list-image-references will provide a list of all the occurrences that have an image associated with them along with details of the image

hamlet --profile msw release list-image-references
| Occurrence                       | Deployment Unit   | Build Reference                          | Code Tag   | Image Format   | Image Source   |
|----------------------------------|-------------------|------------------------------------------|------------|----------------|----------------|
| management-lamb-test-function | lamb | def45678900000 | | lambda | |
| management-apphost-myapp-service | myapp | d66500e2741196b75fc3aed24e35fddad42fa212 | | docker | |
| management-apphost-mytask-task | myapp | d66500e2741196b75fc3aed24e35fddad42fa212 | | docker | |

Event Logging

The logging used when hamlet saves commits to your cmdb has been extended to support a wider range of actions. Events are now stored in an event log stored in your local HAMLET_HOME_DIR. When changes to the CMDB are committed to a repository using the hamlet cmdb commit-changes command the events for the CMDB are collected from the log, formatted as commit messages and saved to the CMDB.

  • add HAMLET_EVENT_DIR
  • event logging for write events

Changes Required

This release had a couple of updates that will require some minor updates to cmdb solutions;

  • On firewall rules, the attribute Type has been renamed to RuleType. This is to fix an issue when the Type attribute was added to all components to set their component type
  • The Solution layer has been removed as it didn't make sense with what layers are intended for. This was also done to make way for being able to define solutions differently in future releases. Any configuration applied on the Solution object in your solutions should be moved to an appropriate layer, most likely the Segment or Environment layer
  • MultiAZ configuration has been moved from the Layers down to the components. A deployment profile was added to all layers that used the old multiAZ configuration option. You should check that your deployments are using MultiAZ deployment options if they need it
  • Backup configuration was removed from the datavolume component. To backup a datavolume we now recommend using the backup component which can handle backing up a much wider ranger of resources

Upcoming

Finally in this release we have deprecated direct usage of the bash executor as the cli should now support all the actions that were available from the bash executor. When running a deprecated command you will be given a warning that the command has been deprecated. If you have questions about which command replaces a deprecated command feel free to reach out in our GitHub discussions.

Full Change Log

Engine

New Features

  • (cdn): add priority to cdn routes
  • (lambda): versioned lambda retention policy (#1989)
  • (alerts): add enable attribute on alerts
  • (lb): define external or internal for lb endpoints
  • (lambda): provisioned executions (#1980)
  • (datavolume): zone control and remove backups
  • (cdn): add support for disabling event handlers
  • add support for HealthCheck Protocol
  • (ec2): add support for zone based deploy control

Fixes

  • add generation contract for unitlist view
  • (runbook): don't include disabled tasks
  • only include active layers based on district type
  • (account): minor fixes for account level aws deployments
  • support regions in state that use refs
  • minor version upgrade control
  • typo
  • typos in attributes

Refactorings

  • (apigateway): authorization models (#1995)
  • testing output handling and test profiles
  • move functions out of setContext
  • update testing to remove solution layer

AWS Engine Plugin

New Features

  • (vpc): support creating security groups without inbound ports
  • (lb): add support for lb backends
  • (datafeed): add support for data streams as a source
  • (cdn): rule priority sorting
  • (rds): event support (#569)
  • occurrence level configuration tagging
  • (lambda): sqs batch control
  • (lambda): versioned lambda retention policy (#574)
  • (alertslack): allow setting runtime with param (#570)
  • (alerts): add filter on alerts for enable status
  • (ec2): support IPAddress groups and LB on same port
  • only include enabled occurrences in suboccurrence processing
  • (lambda): provisioned executions (#559)
  • remove auto state generation for fixutre testing
  • (secretsmanager): add read write support fo secrets
  • (apigateway): mutual TLS attribute (#548)
  • ec2 resource outputs and replace updates
  • (cdn): only add enabled event handlers
  • (cdn): add error for wrong logging region
  • (lb): add support for alb as a network target (#537)
  • add role tag to components if present
  • (ec2): zone based control for instances

Fixes

  • (ecs): tags handling (#588)
  • handle missing link
  • (datastream): typo in attribute name
  • (globaldb): tag function call
  • (sqs): add dlqName back into setup routine
  • handle empty tag sets
  • add backup tags for dds
  • spelling in message
  • (s3): handle notifications for endpoints already deployed
  • casing for MulitAZ attribute
  • format json content for run task module
  • (ec2): fix ordering for cfn init commands
  • (ecs): ensure subnets are always treated as an array
  • (ecs): paramter types for templates
  • typo
  • (lb): use suboccurrence for static forwardning
  • testing updates (#556)
  • various updates from testing
  • log and account processing
  • include Value in getReference
  • (computecluster): general fixes
  • (iam): inline policy creation from policy set (#547)
  • (backupstore): tag based conditions
  • typo in message

Refactorings

  • (apigateway): authorization models (#581)
  • (network): remove baseline components that aren't required
  • network subnet function
  • move test module loading to product layer
  • (iam): limits used for policy splitting (#549)
  • (datavolume): zone filter support for volume mounts

Azure Engine Plugin

  • move test module loading to product layer

Full set of changes: 8.5.0...8.6.0

Bash Executor

New Features

  • detect existing images (#326)
  • add entrance output dir for imagedetails

Fixes

  • spelling in messages
  • three level context properties (#327)

Refactorings

  • sort template objects
  • scope settings search and remove legacy
  • remove deprecated integrator commands
  • remove blueprint doc generation
  • remove reference to slices
  • automation setContext updates

· 9 min read

With our release process bedding in, we've been able to speed up our release process. So here we are with another release with some minor but useful fixes and features.

Key Features

Runbook Tasks for existing processes

In the 8.4 release we added the initial support for RunBooks. They provide a way to run operational tasks on your deployments using the hamlet CLI. Previously this was implemented through dedicated bash scripts for a specific task, in this release we have now replicated these scripts as runbook tasks and included some modules to help with setting them up

  • AWS Module runbook_rds_snapshot adds support for a runbook task that will take a snapshot of a db instance.
  • AWS Module runbook_run_task starts a container task with a provided command and waits for the task to exit. You can provide environment or command overrides as part of running the runbook or provide fixed values. This is useful when you want to run a configuration task such as database migrations as part of your deployment pipelines
  • The baseline AWS module has been introduced which adds runbooks for encrypting and decrypting content using the baseline KMS key.

Each of these modules provides our recommended approaches to using the runbook tasks. You can also use any of the tasks included in these modules as part of your own runbooks

  • engine: (tasks): add basic tasks
  • engine: (ecs): add support for default task compute provider
  • engine: (tasks): extend ssh tasks and add bash command
  • engine: (ecs): add initprocess support for containers
  • aws: add additional runbook tasks and modules
  • aws: add run ecs task support for runbooks
  • aws: add baseline encryption module
  • aws: (ecs): support for ecs exec
  • aws: extended runbooks for access

Backup Components

Backups have now been mapped in to hamlet components with the backupstore and backupsstoreregime components.

The store represents a backup store that will host the backups that have been made. Each store has regimes that define the resources that should have backups in the store. The regime uses link based processing or layer context rules to define your backups.

This allows for a really simple setup for backups if you just want to backup anything that supports backups

Using a component such as

{
"backup" : {
"Type" : "backupstore",
"Lock" : {
"Enabled" : true,
"MinRetention" : 30,
"MaxRetention" : 365
},
"Regimes" : {
"allComponents" : {
"Targets" : {
"All" : {
"Enabled" : true
}
},
"Rules" : {
"nightly" : {
"Schedule" : {
"Expression" : "cron(0 1 * * *)"
}
}
},
"Conditions" : {
"MatchesStore" : {
"Enabled" : true,
"Product" : true,
"Environment" : true,
"Segment": true,
"Tier" : false
}
}
}
}
}
}

I can create a backup store that will backup any resources in my segment, store backups for a minimum of 30 days and a maximum of 365 days. Any new resources added to the segment should be automatically detected and added to the backup schedule. This has been implemented in AWS using AWS Backup. Hamlet helps with the backup setup as we define the tagging structure applied to all components, so we can create simple backup rules using this standardised tagging structure.

  • engine: (backup): Backup support (#1921)
  • engine: (backup): Configuration options (#1926)
  • engine: backup encryption key (#1927)
  • aws: (backup): Initial AWS implementation (#507)

Updates to the CI/CD container extensions

The hamlet and dind extensions have been updated to align with the latest configuration environment variables used in the hamlet CLI;

  • AWS_AUTOMATION_USER has been replaced by the HAMLET_AWS_AUTH_SOURCE and HAMLET_AWS_AUTH_USER settings

  • When using dind as a docker host TLS verification is required at all times

  • Use of the rexray/ebs plugin for docker has been removed as the plugin no longer seems to be maintained and is failing on newer ec2 instances. The recommendation now is to use local volumes and extend the size of your docker volume

  • add support for using local docker volumes for builds

  • refactor hamlet and dind docker extensions

Log Encryption at Rest

Mostly an AWS function but we have added support for enabling log group encryption at rest across all components which create log groups. This is an extension of the LoggingProfile that can be applied across all components to ensure that log encryption is enabled. Adding this in the logging profile makes it an easy set and forget option to enable encryption for logs at rest whenever a new log group is created.

  • engine: (legacy): add encryption at rest support for logs
  • engine: add account layer control over logging
  • engine: add logging profile support for encryption at rest
  • aws: (logs): add support for at rest encryption of cw logs

Engine Input Sanitisation

The underlying engine that we use for hamlet is the freemarker engine, it doesn't have great support for null values imported from JSON objects. As part of our input processing we now strip any null values and their keys from the JSON object.

To help understand how inputs are processed within the engine we now have a new entrance called an inputinfo entrance. This lists all of the input stages and how the processing is assembled for different input providers

  • add inputinfo entrance
  • add null value cleaner to input stages

Bundled Core Engine

The freemarker engine is built as a Java based application that we call as part of the hamlet CLI. This is a bit of a surprise for someone starting with hamlet especially when using our python base CLI. So to help with that the core engine is now distributed with its own Java installation as a bundled command line tool. While this does increase the size of the engines as they are downloaded it does make it a lot easier to install and get going with hamlet.

In this update we have also moved from Java8 to Java11 to catch up with the latest Java releases.

Component Updates

The updates below are the key change log updates from each of the hamlet engine components

Engine: New Features

  • max-age control on bucket content (#1953)
  • (adaptor): add support for alert configuration
  • full names based on district
  • (directory): support for log forwarding
  • (datafeed): compression control for buckets
  • (globaldb): add alerts support
  • (ses): add control over IP access policy
  • (cdn): add origin connection timeout support

Engine: Fixes

  • include component settings in environment

  • district type for district lookup

  • include account in mock filter layers

  • encryption of logs for sms

  • use raw path for settings path prefix

  • (apigateway): handling of null values in definitions

  • prefix handling for shared provider

  • (lb): add required logging profile to lb (#1925)

  • module lookup process

  • (cdn): add logging profile

  • (cd): engine install

  • domain parent handling

Engine: Refactorings

  • asfile ordering (#1955)
  • use local engine definition for testing
  • update deployment group district handling
  • rename district to district type
  • remove task containers from shared provider
  • consolidate link functions
  • remove type based attributes from healthchecks
  • (s3): use recommended process for bucket policy
  • (district): use attributeset for config
  • more specific name part config
  • bootstrap clo processing
  • Account descriptions and placement profiles (#1916)
  • attributesets for components
  • move domain and certificate to reference data

Engine: Others

  • update runtimes for lambda environment

Bash Executor: New Features

  • add support for bundled freemarker wrapper

  • handling of bundled vs jar wrapper

  • add support for providing the district CLI option to templates

  • add docdb support (#312)

Bash Executor: Fixes

  • district default handling
  • sentry release CLI support
  • make output dir for binary only
  • macos run_id and expo ouputs

Bash Executor: Refactorings

  • district rename to district type
  • use weird seperate for user provided paramters

AWS Plugin: New Features

  • use local engine setup for testing
  • (spa): force max-age for config (#530)
  • (adaptor): adaptor alert support
  • (mta): add enable/disable handling on rules
  • (directory): log forwarding support (#517)
  • (s3): backup support (#516)
  • (kinesis): compression support for firehose
  • (globaldb): cloudwatch alarms (#508)
  • (cdn): add support for origin connection timeouts

AWS Plugin: Fixes

  • segment seed fixture value (#535)
  • region lookup for resources
  • (healthcheck): add more entropy to naming of health checks
  • (db): secret lookup for engine setup
  • typo in module
  • (healthceheck): testing changes from type to engine
  • remove use of isPresent for AV setup
  • ipmatch and geomatch for wafv2 (#518)
  • (db): aurora cluster updates
  • (task): kms encrypt parameters
  • efs mount script formatting
  • (db): ingress security group id
  • (sns): add support for encrypted topics
  • (lb): logging profile for WAF logs (#510)
  • (cdn): missing logging profile for waf logging
  • (cdn): logging script for wafv1
  • clean up old if statement
  • (s3): replication validation checking

AWS Plugin: Refactorings

  • align the run task module to task
  • update ecs task configuration after testing
  • move to latest unicycle install process
  • update district to district type on group filter (#534)
  • move ecs container setup to aws provider
  • update iam standard policy name
  • (iam): standard policies for app components
  • (s3): use references for bucket policy
  • backup encryption key (#512)
  • (backup): Configuration options (#511)
  • attribute sets for global configuration

Azure Plugin: Fixes

  • (lambda): align runtimes with latest updates

Refactorings

  • use local engine for testing (#294)

· 11 min read

Another year and another release is here. This release has mostly been housekeeping, with bug fixes and refactoring taking the charge in the changelogs. But it is always nice to have new toys, so we have also included some quality of life features as part of this release post.

· 15 min read

This release post is a long read for a couple of reasons, we missed doing one for the 8.2.0 release, and there have been a lot of new features since our last post.

Similar to our last post this release has focussed on the first 30 minutes of using hamlet, along with a collection of updates to make hamlet easy to use in your existing workflows.

· 15 min read

Another release another stack of changes! The main focus on this release has been on inputs, outputs and the first 30 minutes of using hamlet. The first 30 minutes when working with a new tool is a crucial point for users to see how something works and to get a feel for it. hamlet can be a bit intimidating to start off with so we wanted to reduce this as much as we could.

· 10 min read

Instead of defining the raw infrastructure we need, hamlet works off a solution. The solution outlines the functional components required in the deployment and hamlet then takes this to build out the infrastructure definition that would perform this function.

Combining this with our pluggable architecture we can use the same solution to generate infrastructure definitions in different cloud providers like AWS and Azure. This is an interesting feature itself but in this post we are going to look at another way to use this data, diagram generation.

· 12 min read

Static websites are everywhere nowadays, alongside them on their rise in popularity are the numerous app frameworks like Jekyll, Gatsby and Docusaurus that allow anyone to spin up and deploy documentation, a blog or simple website, often for for little or no cost.

· 11 min read

hamlet has its own opinions on how things should be done. These opinions are formed by our community based on best practice ( including the best way to workaround things ) to get you working in the cloud as efficiently as possible. Sometimes you might want to try out something new that we don't support or a component we offer isn't quite right. In this series of posts, Prototyping Deployments you should get an idea of how hamlet helps with these deployments and integrates with the rest of your solution.