Skip to main content

v5.5.0 ( June 2019 )

ยท 7 min read

Welcome to v5.5.0 of codeontap which focuses on adding a collection of new components and reworks our segment unit components

Features

Expo Mobile App build and deploy

Adds support for the build and deploy of Expo based mobile apps

This introduces a new component mobileapp which is used to control how the app is deployed. Builds are performed as publish tasks since they are required for each environment rather than our standard build once deploy many approach.

Settings are supported and are made available as constants to the app

Commits

  • Mobile app - Add deployment (#686)
  • Add support for multi sdk builds (#689)
  • expo release channel(#700)
  • mobile app settings(#718)

Segment units refactoring and updates

The core segment units ( network/vpc, cmk and s3 ) have all been refactored into our occurrence based structure A few features have also be introduced as part of this

  • Networking has now been broken up into two components
    • network - This consists of a private network that is made up of subnets, route tables and high level network ACLs for access control between subnets
      • gateway - A gateway attaches to a network route table and provides an exit point from the network. We currently support 3 engines
        • nat - outbound NAT gateway
        • igw - internet gateway which provides public internet access for the network
        • vpcendpint - direct connections to cloud provider API endpoints services which are generally available at public endpoints. This includes support for endpoint and gateway vpc endpoints
  • A new component has been introduced as the baseline component. This component is intended to provide a collection of different resources which must be deployed for any segment to function. This includes the following subcomponents
    • DataBuckets - Shared buckets which can be used as standard storage locations for application data and infrastructure logging when a service doesn't use a logging facility ( AWS cloudwatch ). These were the old s3 segment units
    • Keys - Shared private keys that are used across a segment. This includes kms keys, ssh keys and CDN keys ( OAI keys for cloudfront to S3)

All segment level unit changes have been made backwards compatible but some extra units need to be added to ensure that it aligns with the new components

When deploying segment units the following units should now be used

Existing deployments - baseline, iam, lg, vpc, vpcendpoint, eip, nat, ssh

New deployments - baseline, iam, lg, vpc, igw, eip, vpcendpoint, nat, ssh

All network components are now optional, some dependencies do apply between the components ( igw must be deployed if using nat in AWS)

Commits

  • Refactor networking into components (#658)
  • segment cmk into baseline (#789)
  • Refactor Segment S3 and S3 object notifications (#706)

Data Volumes

Data volumes provide a compute independent persistent storage volume which can be attached to compute components, starting with Ec2 and ECS. Volumes also have automated backup policies which will manage snapshots of the volumes

Commits

  • Adds support for persistent data volumes on ECS
  • extends datavolume support to ec2 (#701)
  • Add automated snapshots with SSM Automation (#703)

Config Stores

Config stores provide a DynamoDb based configuration lookup table which is populated using hamlet links and settings. This allows for components to dynamically lookup configuration rather than having to redeploy a component to update its configuration

Commits

  • Config Stores (#722)
  • config tables - secondary keys

Data Feeds

Data feeds are a new component which provides continuous data streams to backend storage services. For AWS this is based on kinesis firehose. The datafeed component must have a destination link ( at the moment an ES component) and can optionally include logwatchers which link to components and create log subscriptions to logs generated by the component. These are then sent to the destination

When processing cloudwatch logs you need to also include a link to a lambda function which can process the log message and generate a record ( cloudwatch logs are base64 encoded and gzipped when sent in the subscription). We have created a couple of generic log processing lambda functions which are available here

Commits

  • Elasticsearch logwatcher using kinesis data firehose (#736)
  • Data feed component (#788)

Service Registries

Service registries provide a DNS or API based lookup service which components can register their network access information with. The initial release adds support for private registries which use AWS Cloud Map to create a private DNS zone, which can be used for access between services in the same segment. Currently services and lb components can link to Service registries to provide their network details

Commits

  • Service registries (#794)

User pool federation and client management

Based on some more industrial strength usage of userpools we have made some updates to how user pools are deployed

  • Clients are now a subcomponent of userpools so that you can deploy multiple clients with different authentication flows, and scopes. This is handy if you have a server side authentication and mobile app authentication where flows and scope requirements might differ
  • Added support for user pool federation which allows for a userpool to dynamically create users in the pool based on another auth provider. This is available using the authprovider subcomponent on user pools

Commits

  • User pools with multiple clients (#724)
  • Userpool Hosted UI own domain name (#725)
  • Cognito Userpool federation (#730)

WAF Updates

Extends our support for WAF to include application inspection rules along with IP and country whitelisting This includes

  • Adding profiles for rules and conditions
  • Adding support for regional endpoints ( API Gateway )
  • Add ability to define a collection of rules and conditions as waf security profile
  • Add basic starter profile which aligns with the OWASP 2017 Top 10

Commits

  • Regional WAF Support (#732)
  • Expanded WAF Support for API Gateway (#745)
  • Update WAF support for SPA (#748)

API Gateway Updates

A few different updates have been added for API Gateways

  • The publish option has been deprecated due to issues we had with S3 bucket name lengths. To replace this functionality we have introduced spec publishers which work with the contenthub component to generate a api specification which includes server endpoints and authentication details and copies it to a content hub location. This is intended to work with our newest hamlet family member townplanner which provides a centralised document publishing platform.
  • Added support for openapi3 specification integration with our api gateway build process
  • Some extra features in our apigateway enrichment process

Commits

  • ApiGateway: spec publishers (#763)
  • Api Gateway - basepath control (#795)
  • apigw: add content handling support (#781)
  • apigw: openapi3 support for security srvs (#782)

ECS Fargate support

Support for running ecs services and tasks on AWS Fargate

Commits

  • ECS Fargate hosting (#704)
  • Fargate scheduled tasks (#716)

Extra features

  • Domain inclusions
  • SPA Path based routing to LB (#749)
  • update freemarker to 2.3.28 (#784)
  • suppurt multi-template location
  • jenkins fragment memory configuration (#707)
  • Processor profiles for occurrences
  • Local build commit is preferred over shared (#778)

Refactor

RDS Replacement options

We've added some extra configuration options around RDS restoration and replacement processes to speed things up when you are working with snapshot based rds instances and aren't as worried about the data in the database

Commits

  • RDS Template Polcy management
  • RDS resotre from rds dataset
  • rds replace through delete

Other changes

  • Remove legacy support for containers (#714)
  • Force removal of "containerId" use (#728)
  • templates: remove segment unit control
  • deployment profiles with instance version (#779)
  • Templates should not depend on tier and component objects (#793)
  • Lambda edge functions now use nodejs8.1

Fixes

  • add priority ordering to cfn init scripts (#702)
  • skip network lookups for segment iam
  • dyanmoDb arn formatting (#734)
  • fix path for sensitive settings
  • remove alarms when hibernating (#739)
  • Conditionally create logmetric resources (#752)
  • Data pipeline resource creation cleanup (#754)
  • Disable predefined resources which are not required
  • apigw: use cogntio api arn (#783)
  • s3 lifecycle allow for numbers in rules (#792)