Skip to main content

v8.4.0 ( January 2021 )

ยท 11 min read

Another year and another release is here. This release has mostly been housekeeping, with bug fixes and refactoring taking the charge in the changelogs. But it is always nice to have new toys, so we have also included some quality of life features as part of this release post.

Setting Configuration

Settings are used in hamlet to provide application level configuration. They are used for providing environment variables, complex configuration options to components and making extensions dynamic. Configuring settings in previous releases has required Setting Namespaces, which use a directory structure approach for setting look-ups driven by properties of a component (such as their name, or a custom namespace). This is really useful when dealing with large configuration requirements as you can share configuration across components and have dedicated files for your configuration vs the solution definition.

But sometimes you just want a simple way to provide a couple of environment variables and get going. With this release we've added support for defining settings in your component solution configuration. All components now have a Settings attribute where you can add Key value settings that will automatically be added to component environment variables when they are deployed.

For more details on using settings see the settings how to guide

Secret Engine Referencing

Secret management is an important part of any deployment and ensures that appropriate access control is maintained throughout your deployment, including in the cloud provider consoles. In this release we have extended support for native secrets management integrations provided by cloud providers. The secretstore component now supports creating its own secrets and you can choose to generate the secret or provide your own.

The database component and queuehost components which both support integrating with the secretstore now support using predefined secrets as well. This allows you to control the password for these components yourself while still using the cloud provider's native secrets management.

Finally, for the service and task components we now support using the container orchestrator to manage secret injection. Using this feature, you provide the link to a secret that you want the service or task to use and the orchestrator will retrieve the value and set it to the environment variable you want to use. This avoids having to add custom logic into containers to decrypt the secrets.

Run Books

Operational and troubleshooting tasks are key parts of any deployment and having standard approaches to perform them ensures you can manage your deployments efficiently. In this release we have added support for Runbooks to provide a method to define a list of tasks that will be run by the hamlet executor. This means that the tasks will run from wherever you are running the hamlet commands from, instead of in the deployment itself.

The tasks are defined as part of the hamlet engine and describe the parameters for each task, as well as a description of what should be completed. The executor is then responsible for implementing these tasks in line with the description.

To get started on how Runbooks work we have included some modules in the AWS plugin for some common tasks that we do as part of our deployments. To show how Runbooks work we will look at the ssh-session Runbook that is included in the ssh_bastion module.

{
    "ssh-session" : {
        "Description" : "Starts an interactive SSH session with the bastion host using the baseline ssh key",
        "Type" : "runbook",
        "Engine" : "hamlet",
        "Steps" : {
            "aws_login" : {
                "Priority" : 5,
                "Extensions" : [ "_runbook_get_provider_id" ],
                "Task" : {
                    "Type" : "set_provider_credentials",
                    "Properties" : {
                        "AccountId" : {
                            "Value" : "__setting:ACCOUNT__"
                        }
                    }
                }
            },
            "ssh_key_decrypt" : {
                "Priority" : 10,
                "Extensions" : [ "_runbook_get_region" ],
                "Task" : {
                    "Type" : "aws_decrypt_kms_ciphertext",
                    "Parameters" : {
                        "Ciphertext" : {
                            "Value" : "__attribute:ssh_key:PRIVATE_KEY__"
                        },
                        "EncryptionScheme" : {
                            "Value" : "__attribute:ssh_key:ENCRYPTION_SCHEME__"
                        },
                        "AWSAccessKeyId" : {
                            "Value" : "__output:aws_login:aws_access_key_id__"
                        },
                        "AWSSecretAccessKey" : {
                            "Value" : "__output:aws_login:aws_secret_access_key__"
                        },
                        "AWSSessionToken" : {
                            "Value" : "__output:aws_login:aws_session_token__"
                        }
                    }
                },
                "Links" : {
                    "ssh_key" : {
                        "Tier" : "mgmt",
                        "Component" : "baseline",
                        "SubComponent" : "ssh",
                        "Type" : "baselinekey"
                    }
                }
            },
            "start_ssh_shell" : {
                "Priority" : 50,
                "Task" : {
                    "Type" : "ssh_run_command",
                    "Parameters" : {
                        "Host" : {
                            "Value" : "__attribute:bastion:IP_ADDRESS__"
                        },
                        "Username" : {
                            "Value" : "ec2-user"
                        },
                        "SSHKey" : {
                            "Value" : "__output:ssh_key_decrypt:result__"
                        },
                        "Command" : {
                            "Value" : "/bin/bash"
                        }
                    }

                },
                "Links" : {
                    "bastion" : {
                        "Tier" : "mgmt",
                        "Component" : "ssh"
                    }
                }
            }
        }
    }
}

This Runbook performs the required tasks to open an ssh session on a bastion host. This includes getting the ssh key that we generate as part of baseline deployments, decrypting it using KMS and then opening the ssh session.

Runbooks support a look-up syntax for their parameter values to make it easier to pass results from step to step or to access the context of your hamlet deployment. They use the syntax of __<source>:<id>:<additionalid>__ and hamlet currently supports the following sources:

  • input: is an input provided to the pipeline by a user
  • output: is an output provided from another step
  • attribute: is the attribute of a link that has been defined on the step
  • setting: uses settings provided to the runbook step

To use the Runbook we use the hamlet CLI.

Show the available Runbooks:

hamlet task list-runbooks
| Name                               | Description                                                                        | Engine   |
|------------------------------------|------------------------------------------------------------------------------------|----------|
| management-ssh_ssh-session-runbook | Starts an interactive SSH session with the bastion host using the baseline ssh key | hamlet   |
| management-db_pgdump-runbook       | Creates a pg_dump of the database and saves it to a local file path                | hamlet   |

Run a Runbook:

hamlet task run-runbook -n management-ssh_ssh-session-runbook
[*] management-ssh_ssh-session-runbook
[*]   Starts an interactive SSH session with the bastion host using the baseline ssh key

[-] Running contract stage aws_login
      Step: aws_login - Task: set_provider_credentials
[-] Running contract stage ssh_key_decrypt
      Step: ssh_key_decrypt - Task: aws_decrypt_kms_ciphertext
[-] Running contract stage start_ssh_shell
      Step: start_ssh_shell - Task: ssh_run_command

---- SSH Session | host: 1.2.3.4 | user: ec2-user ----

[ec2-user@ip-10-0-245-144 ~]$

Each step shows what is being run as well as any output that is generated and the CLI will pass through interactive sessions as required. Inputs, when required, can be added as key=value arguments on the CLI command.

If you'd like to include the Runbooks from the list-runbooks command in your existing solutions they are available in the modules:

Test tooling Updates

With deployment testing we have made a couple of updates to make testing easier and provide better control over the configuration of tests:

  • cfn_nag has been replaced with Checkov which provides security best practice auditing for a wide range of declarative infrastructure templates including CloudFormation and ARM templates. Checkov is also Python-based so we can include it as part of our CLI installation.
  • When configuring testing tools you can now provide configuration options to the testing tools. This allows you to disable particular checks or enforce the use of a particular framework format when working with Checkov.

Stack Management Updates

AWS Stack management now handles failed creations and will remove the existing stack before attempting a new creation. This avoids having to go into the console when a deployment fails on the initial creation.

Azure deployments have been updated to more closely align with the changes to the AWS stack management processes to ensure that a consistent approach is used across the two providers.

Executor Updates

We've made a range of changes to both the CLI and bash backend to make it easier to work with hamlet:

  • dos2unix has been removed from bash scripts. If this is required, it will need to be managed outside of hamlet from now on. Git mostly handles this process anyway.

  • We've updated the CLI framework that we use (Click) to its latest major release and as part of that update made the autocomplete setup a dedicated command. To enable autocomplete run the following command:

    hamlet autocomplete <your shell of choice>
    

    It currently supports bash, fish and zsh. After running the command you may need to restart your shell for it to take affect. Note that bash is still used by all hamlet commands in the background, but you can use any shell you like when making calls through the hamlet CLI.

  • Location directory checks and authentication for the automation scripts have been removed from the bash executor. This means you can run hamlet commands from anywhere as long as the context is provided.

  • MacOS support has had a couple of updates to ensure that commands in the bash executor align with what is available in the OS

    • Sleep commands have been updated to only specify the time as a number
    • Run ID generation which relied on using dd has been replaced with the built-in $RANDOM built-in bash function + base64 encoding
    • Testing of the CLI for development no longer uses mknod which is considered a privileged command in MacOS
  • If your are using the URL or container registry image sources you can skip pulling the image by setting the env var HAMLET_SKIP_IMAGE_PULL=true when running a deployment. This is handy when you are verifying a deployment.

Component Updates

In this release we added a couple of components and made some minor updates to align naming:

  • outbound MTA: The MTA component now supports both inbound and outbound mail handling using the same rule-based approach for each direction
  • client VPN support: client VPNs define a way for users to access a private network through an authenticated encrypted session from their clients
  • LB Conditions: load balancer conditions have been extended to support a range of different conditions, including HTTP standard details like headers and query strings along with IP addresses
  • EFS renamed fileshare: the EFS component has been renamed fileshare to provide vendor agnostic naming for the component and allow for implementing other engines in the same fileshare functionality.

AWS

The AWS plugin has had a range of updates to support different services on our current components:

  • wafV2 for AWS: For all WAF configuration options you can now provide a Version attribute to enable WAFv2 over the current classic WAF
  • Dynamic Path support for kinesis firehose allows you to control where a record is placed in s3 based on details found during processing of a record
  • ECS Service/Task now supports specifying the number of GPUs required for a Container
  • MTA inbound and outbound support has been implemented
  • Filters can be used on topic subscriptions to control the messages sent to a particular subscription
  • For AWS plugin development we have now enabled cfn-lint across all of our test generation to ensure that we comply with cfn-lint
  • The directory component now supports AWS AD Connectors which act as proxy services to existing AD DS installations
  • EFS now supports creating FSX for SMB file shares
  • When deploying a directory component a route53 resolver is added to route DNS requests from the standard AWS DNS server to the AD servers. This removes the need for creating DHCP changes or manually configuring your DNS servers on instances
  • Application LB now supports additional conditions when applying rules
  • CFN-HUP support has been added for Linux and Windows based EC2 instances to re-run cfn-init scripts on CloudFormation updates

Azure

The Azure plugin has had some updates focusing on networking and directories:

  • It adds support for the directory component using Azure AD Directory Services which creates an AD DS installation based on your Azure AD
  • The network component has been refactored significantly to provide more control over network ACLs and the provisioning of network resources
  • Site to Site VPN support has been added to the Gateway component

Major Fixes

A lot of bugs were taken care of during this release. These can be found in the change logs, but the following are the most significant fixes:

  • s3 versioning without lifecycle management: When you enable versioning on an S3 component, lifecycle rules are automatically enabled to remove files from the bucket based on the default retention period of the environment. As a result objects won't be visible to applications and appear to have been deleted. Since enabling versioning keeps the files after they are deleted, data will not be lost but files may need to be reverted to existing versions. Please check that your S3 buckets in AWS are working as expected.
  • orphaned deployments are always first: A bug related to orphaned deployment processing meant that orphaned deployments were not being found. This has now been resolved and orphaned deployments will now be processed before other deployments to ensure they don't cause issues when running updates.

Full Change Log

To see the full set of changes check out the CHANGELOGS for each of the hamlet services below: