Another release another stack of changes! The main focus on this release has been on inputs, outputs and the first 30 minutes of using hamlet. The first 30 minutes when working with a new tool is a crucial point for users to see how something works and to get a feel for it. hamlet can be a bit intimidating to start off with so we wanted to reduce this as much as we could.
Engine Updates
Input and output handling was where we spent most of our time in this release and as a result we now have a much more flexible engine that can handle a lot of different output generation processes along with a pipeline based input processing approach.
Input Handling
Previously our input handling was processed once and each provider could offer specific types of input during this initialisation process. With our new input processing there is now an input pipeline which providers can add their own stages and steps into.
The inputs provided are stored as part of a stack and output look up processing now performs lookups on request instead of relying on fixed input stores. The main reason for this change is to handle to change the state of inputs during processing, such as cross product lookups or looking up the state of different accounts. There is more work underway in this space but the foundations are now in place to support this.
Whatif input source
To test out the new input pipeline we have introduced a new input source, the whatif input source. This input source uses input stages from our mock input source which provides fixed responses to outputs when they can't be found with our composite input source which looks up outputs from a CMDB. The whatif input source allows you to generate outputs for components which aren't deployed yet and see what they look like.
To use this feature, from your segment cmdb run your hamlet commands with the top level option -i whatif
This is really useful when you want to test a new component that has dependencies on a network or other components and want to see what is generated and how it integrates with your solution.
Output Processing
Writing and naming outputs is now part of the hamlet engine. When the engine writes an output it works through a pipeline of output handlers which determine how we write the output.
Managing outputs in the engine we can write to different locations, write outputs in different formats and handle writing outputs to the console as well as files depending on the configuration in the cmdb.
We can now write multiple outputs in one call to the engine instead of having to call the engine for each output. This has improved performance in the engine output processing and now means that generating 2 outputs for a deployment vs 10 is roughly the same speed
YAML output generation
Along with the engine processing performance improvements moving to engine output processing means we can now generate YAML outputs as well as JSON outputs. This allows us to generate configuration for services that only support yaml for configuration such as Jenkins config as code and docker-compose.
Logging Updates
Now that the engine can write outputs whenever it feels like we have introduced better support for logging during the generation process. Log messages are no longer included in the outputs themselves which means we can include warnings or other information during generation without breaking the output itself. This includes adding a compact log format that will be used for non fatal log messages so you can see the warnings without them flooding the console.
Blueprint Validation
A new entrance is available which will validate your blueprint against the schema for the overall blueprint. This allows you to make sure that the configuration you've provided matches what you wanted and that it is working in hamlet as you expect.
See https://docs.hamlet.io/docs/learn/validate for more details on how to use it
Compute Tasks
Virtual machine configuration is now available as part of your solution. This includes changing the OS, virtual machine image and even the bootstrap scripts that are run. This power does have some responsibility. Components are defined based on a specific function, a containerhost component is a container host based on virtual machines, a bastion needs to provide access into the environment. So while you can configure your virtual machines components will define required ComputeTasks in your bootstrapping process.
The compute task itself is just a description of what needs to be done. Using extensions and the computeTask extension macro you define the bootstrap script that completes the task. Once you've completed these tasks, such as installing latest updates or registering with a load balancer you can add your own extensions to perform tasks that you need.
Required Links
Links between components are a big part of hamlet and our current approach has been to handle links to components which aren't deployed in the background. This generally meant that we failed silently and used empty values where the link attribute would be included. You can now control this behaviour to manage how a particular link is handled.
When defining a link you can now add the ActiveRequired attribute to the link. Setting this to true means that if the component that is being linked to hasn't been deployed the process will fail. The default action is now to log a warning message but continue with template generation.
Image Sourcing
We've continued on our support for configurable image sourcing and now provide this for:
- apigateway
- spa
- containertask ( task )
- containerservice ( service )
- adaptor
- template
With some of these ( apigateway, adaptor ) we also identified the need to disable external images. This allows you to completely define these components within hamlet and removes the need to rely on external providers. For the adaptor component this is a really powerful extension point within hamlet and allows you to run scripts or completely custom deployments as part of hamlet. Stay tuned for a blog post on this soon.
Plugin Updates
We welcomed a new plugin into our official providers this release, the cmdb provider. This provider will integrate with our new input and output processing and take over for reading and writing to a CMDB. This is currently in beta and will be enabled in a future release.
AWS
User defined ec2 images
We now support the ability to specify an AMI image for your ec2 based components. This includes an explicit AMI and the use of SSM parameter store. The SSM parameter store allows you to access the aws public parameters which publish the latest image id for each region. This ensures you are always using the latest image for a given OS.
AWS linux 2 support
With the introduction of compute tasks we can now support different operating systems that can be configured as part of your solution. Now that awslinux1 is in maintenance mode we wanted to make the migration to awslinux2 easy. Using compute tasks and SSM Parameter image config we have added a deployment profile which will handle all the config required to use awslinx2 for an ec2 based component. To use the deployment profile, add the deployment profile _awslinux2 to any component that you want to run on awslinux2. If you want to apply it across the board you can also set this profile at the environment, product, account or tenant level.
All of our components have been tested with awslinux2 and our provided compute task extensions support both awslinux1 and 2.
AutoScale Group lifecycle events
We now include a default autoscale group lifecyle event for all autoscale groups. Along with this we also have a required compute task to complete the lifecycle event. This ensures that instances started during autoscale events complete their bootstrap processes and if they don't, they are replaced. This prevents zombie instances from spoiling a cluster.
ECS Updates
We've included support for some of the latest features in ECS for our container hosting.
- Capacity Providers - are now fully supported on services as compute providers. When using the ec2 compute provider this allows for ecs to manage the autoscaling of the ec2 instances in the cluster to meet the required capacity of the cluster. When starting a new task as part of the service, the task will wait for the capacity provider to start an ec2 instance if there aren't any available. Before capacity providers the task start would fail if there wasn't capacity
- Circuit Breakers - We now include ECS service circuit breakers by default to ensure that deployments roll back if the task can not be started properly. Before this the service update in cloudformation would run indefinitely, until you manually intervened.
Health Check Component
The AWS provider now supports deploying healthchecks. Health checks are used to provider transaction monitoring of endpoints to ensure your system is working properly. We have implemented two types of checks, simple and complex.
Simple checks use the Route53 Health check service which performs basic pings of an endpoint and reports the response received.
Complex checks use CloudWatch canaries to run scripted health checks using lambda and can report back health status along with screenshots of what happened. These are great for testing full processes for web apps.
Both support Cloudwatch alarms through the Alerts Configuration.
Diagrams
You can now define diagrams as part of your solution. The plugin provides the type of diagrams, such as solution overview, resources and you can select what is included as part of the diagram. At the moment we allow filters based on component type, resource types and through links to explicit components. This allows you to handle large solutions or generate diagrams for particular parts of your solution.
An example of a solution overview for all components would be
{
"Diagrams" : {
"overview" : {
"Type" : "solution",
"Title" : "All Components",
"Description" : "A solution level diagram of all components in the solution",
"Rules" : {
"allComponents" : {
"Action" : "Include",
"Policy" : "ComponentType",
"policy:ComponentType" : {
"Types" : [ "*" ]
}
}
}
}
}
}
We've also included a module of some basic diagrams to get you going called overviews. Include this module in your solution along with the diagrams plugin to start using them. The hamlet cli also supports generating all of your defined diagrams from a single command and output the generated images to a nominated directory. This is great for including the diagrams in your documentation.
hamlet cli
The hamlet cli is our python based cli utility which wraps around our current bash scripts and the engine.
pypi release
You can now get the hamlet cli from pypi using pip. All changes are deployed as pre-releases and tagged releases like this one will be the default version.
To install the cli run
pip install hamlet
if you want the latest changes to the cli
pip install --pre hamlet
Profiles and district configuration
In hamlet we call the tenant, account, product, environment and segment that you are working in a district. This district you are working on is generally set by your current directory. In the cli you can now control the district you are working in using command options or a configuration file.
To set the district using command options:
hamlet --root-dir ~/mycmdb/ --tenant mytenant --account account1 --product product1 --environment env1 --segment seg1
Since we also include the root-dir you can run this command from anywhere without being in the CMDB
This can be a bit long everytime you run a command, so you can also create profiles as part of a config file. The default location of the profile for is ~/.hamlet/config
In the file you can specify a profile with each of the configuration options above
[profile:disrict1]
root_dir=~/mycmdb/
tenant=myteant
account=account1
product=product1
environment=env1
segment=seg1
Then to use this profile in your hamlet commands
hamlet --profile district1 deploy list-deployments
This will then use the profile in your config file. If you want to override any of these values set them using command line options. This is great for managing a lot of deployments at time and saves having to find the account each time you run a command
Deploy commands
The deploy command group is a core part of the hamlet cli and manages all things deployment related for your product.
Test deployments
The test-deployments command uses the TestCases and TestProfiles defined within your district to run tests on the generated outputs. This makes testing really easy and uses the same pattern based process for finding deployments that we use for the other deployment commands.
To test all your deployments
hamlet deploy test-deployments
To test the application level units
hamlet deploy test-deployments --deployment-group application
and the output looks like this
[*] Creating deployments:
[-] segment/baseline
[-] segment/vpc
[-] segment/igw
[-] segment/vpcendpoint
[-] segment/nat
[-] segment/aws-queuehost-secretstore
[-] segment/ssh
[-] segment/aws-bastion-base
[-] solution/aws-healthcheck-lb
[-] solution/aws-lb-app-https
[-] solution/aws-ec2-base
[-] solution/aws-ecs-base
[-] solution/aws-filetransfer-base
[-] solution/aws-s3-base
[-] solution/aws-s3-notify
[-] solution/aws-s3-replication
[-] solution/aws-s3-replication-external
[-] solution/aws-queuehost-base
[-] solution/aws-db-postgres-base
[-] solution/aws-db-postgres-generated
[-] application/aws-apigateway-base
[-] application/aws-computecluster-base
[-] application/aws-healthcheck-complex-base
[*] Testing deployments:
Test session starts (platform: linux, Python 3.8.8, pytest 6.2.4, pytest-sugar 0.9.4)
cachedir: .pytest_cache
rootdir: /tmp/tmptyct45p2
plugins: sugar-0.9.4, django-4.2.0
collecting ...
test_deployments.py::test_aws_apigateway_base_app_apigatewaybase_apigateway_apigatewaybase ✓ 6% ▋
test_deployments.py::test_aws_bastion_base_mgmt_bastionbase_bastion_bastionbase ✓ 12% █▍
test_deployments.py::test_aws_computecluster_base_app_computeclusterbase_computecluster_computeclusterbase ✓19% █▉
test_deployments.py::test_aws_db_postgres_base_db_postgresdbbase_db_postgresdbbase ✓ 25% ██▌
test_deployments.py::test_aws_db_postgres_generated_db_postgresdbgenerated_db_postgresdbgenerated ✓ 31% ███▎
test_deployments.py::test_aws_ec2_base_app_ec2base_ec2_ec2base ✓ 38% ███▊
test_deployments.py::test_aws_ecs_base_app_ecsbase_ecs_ecsbase ✓ 44% ████▍
test_deployments.py::test_aws_filetransfer_base_app_filetransferbase_filetransfer_filetransferbase ✓50% █████
test_deployments.py::test_aws_healthcheck_complex_base_app_healthcheckcomplexbase_healthcheck_healthcheckcomplexbase ✓56% █████▋
test_deployments.py::test_aws_lb_app_https_elb_httpslb_lb_httpslb ✓ 62% ██████▍
test_deployments.py::test_aws_lb_app_https_elb_httpslb_lb_validation ✓ 69% ██████▉
test_deployments.py::test_aws_queuehost_base_app_queuehostbase_queuehost_queuehostbase ✓ 75% ███████▌
test_deployments.py::test_aws_s3_base_app_s3base_s3_s3base ✓ 81% ████████▎
test_deployments.py::test_aws_s3_notify_app_s3notify_s3_s3notify ✓ 88% ████████▊
test_deployments.py::test_aws_s3_replication_app_s3replicasrc_s3_s3replica ✓ 94% █████████▍
test_deployments.py::test_aws_s3_replication_external_app_s3replicasextrc_s3_s3replicaext ✓ 100% ██████████
Results (11.12s):
16 passed
Dry run
You can now add the --dryrun option to the run-deployments command. Before running each deployment the dryrun functionality of the provider will provide a list of the changes that will occur during the deployment. For AWS this is through Cloudformation Change sets and for Azure we use the whatif option.
Combining this with the --confirm options means you can find out what your deployment will do and make a decision if you should continue.
hamlet deploy run-deployments --dryrun --confirm
Setup command
We've added support for running the setup command for installing plugins for your product. This saves you having to change context and using the bash provider
It's nice and easy, from your segment cmdb
hamlet setup
This will install the plugins that have been defined in your product
Component command group
We have added a new command group called the component group. This group contains commands for components and occurrences. The first commands in this group are for finding out the state of occurrences.
list-occurrences shows you all occurrences and suboccurences that are part of your district
hamlet -i mock component list-occurrences
| TierId | ComponentId | Name | Type |
|----------|---------------|--------------------------------|--------------------|
| mgmt | baseline | management-baseline | baseline |
| mgmt | baseline | management-baseline-opsdata | baselinedata |
| mgmt | baseline | management-baseline-appdata | baselinedata |
| mgmt | baseline | management-baseline-ssh | baselinekey |
| mgmt | baseline | management-baseline-cmk | baselinekey |
| mgmt | baseline | management-baseline-oai | baselinekey |
| mgmt | ssh | management-ssh | bastion |
| mgmt | vpc | management-vpc | network |
| mgmt | vpc | management-vpc-internal | networkroute |
| mgmt | vpc | management-vpc-external | networkroute |
| mgmt | vpc | management-vpc-open | networkacl |
| mgmt | igw | management-igw | gateway |
| mgmt | igw | management-igw-default | gatewaydestination |
| mgmt | nat | management-nat | gateway |
| mgmt | nat | management-nat-default | gatewaydestination |
| mgmt | vpcendpoint | management-vpcendpoint | gateway |
| mgmt | vpcendpoint | management-vpcendpoint-default | gatewaydestination |
You can then use the name of the occurrence to get information about the occurrence
hamlet -i mock component describe-occurrence -n management-baseline
{
"Configuration": {
"SettingNamespaces": [
{
"Key": "management-baseline",
"Match": "partial"
},
{
"Key": "management-baseline-baseline",
"Match": "partial"
},
{
"Key": "mgmt-baseline",
"Match": "partial"
},
{
"Key": "mgmt-baseline-baseline",
"Match": "partial"
},
{
"Key": "baseline",
"Match": "exact"
}
]
...
Here you will get the full occurrence. You can use the --query option to use jmespath queries on the JSON response. We have also included some predefined queries.
Commands:
attributes Predefined query: The attributes for the occurrence
resources Predefined query: The resources of the occurrence
setting-namespaces Predefined query: The setting namespaces for the occurrence
solution Predefined query: The evaluated solution configuration for the occurrence
Full Change Log
So this has been a pretty big set of changes that we think help using hamlet and provide a lot of flexability in what you can do. To see the full change list head to each repos change logs