Bokbasen CI/CD Pipeline – Case Study

Bokbasen provides a wide range of services to all parts of Norway’s publishing community.

The company builds and maintains the Den norske Bokdatabasen catalogue containing data about all Norwegian publishers.  By making use of this single centralized register, the industry can easily make use of the data – in online stores, bookshops, libraries and schools – instead of spending time gathering it, as is the focus in many other countries.

When e-books entered the market, Bokbasen was quick to build the infrastructure to support them, including a streaming service for audiobooks. AWS partner Orange Business has been hosting and managing operations for Bokbasen in its private cloud since 2012.

In 2017 Bokbasen started utilizing AWS for storing and transcoding of audio files.

In 2018 Bokbasen migrated their public website to AWS.

In 2019, Bokbasen set out on a new mission to digitize books and learning materials, beginning with Digitalelev, a product for management and procurement of digital learning materials in schools, and Allvit, a platform for distribution of textbooks and resources for higher education.

The Challenge

A key challenge was to deliver these new products, with an IT department of just five employees. The solution was to use a combination of flexible managed services from AWS, in combination with experienced subject matter experts from Orange Business.

Bokbasen chose to use a container based microservice architecture to enable their new services to be easier and more agile to extend with new functionality, while also allowing the component services to scale independently. Orange Business worked with Bokbasen to deploy and manage DevOps Pipelines, which seamlessly automate the process of releasing updates and new functionality.

DevOps

DevOps represents a culture shift that integrates the team that authors code, with the team that operates it, forming a single team.  Instead of each software release containing many, potentially disruptive changes that are all released at once, teams are encouraged to release small changes, more frequently, thus reducing the impact of change and making it easier to test and release.

Orange Business engages with customers to support their DevOps business and technology transformations.

The customer chose to use a deployment strategy that involved:

  • Building a single artifact for all environments
  • Release early, release often
  • Use a rolling deployment to gradually replace or upgrade services
  • Always fix defects by rolling forward to a new release

It was important for Bokbasen that their own DevOps team had the power to deploy consistently to all environments, including production.

Solution

Infrastructure as Code

Utilizing Infrastructure as Code technology (IaC), Orange Business and Bokbasen can automate the deployment of infrastructure, application services and CD/CD pipelines across environments.

HashiCorp Terraform is used to deploy Bokbasen’s infrastructure. Self-service terraform templates are being used to provide quick and easy creation of new pipelines.

Many great IaC modules from the Hashicorp Terraform registry, and from Orange Business, have simplified the transition. The use of IaC ensures a consistent and repeatable method of deploying and managing infrastructure across environments.

Spring Cloud Config and AWS Systems Manager Parameter Store are used to store configuration and keep secrets such as security credentials encrypted and audited.

Deployment Pipelines

Orange Business Professional Services worked with Bokbasen to develop pipelines to build application artifacts and deploy microservices across Bokbasens Test, Stage and Production accounts, integrating with the following main AWS services:

  • Lambda
  • Fargate
  • Batch
  • API-Gateway
  • Static websites on S3 (utilizing CloudFront CDN)

When a member of the DevOps team commits to selected branch, the pipeline is triggered, which builds artifacts, such as docker images that are part of the service and submits those images to the container registry.  The pipeline then deploys the container to the Test environment. A smoke test using RunScope is launched and the results of the test are manually verified by the DevOps team before approving the release to be deployed to stage, where the same smoke test and additionally verification tests are undertaken before approving deployment in Production. DevOps have read-only access to stage and production environments. The pipelines are allowed cross-account deployments assuming IAM roles.

Database schemas are also modified as part of the deployment pipelines using Flyway.

Similar pipelines are created for CI/CD of batch jobs, lambda functions and S3 hosted websites.

CI/CD: Fargate hosted containers

CI/CD: Lambda functions

CI/CD: Batch jobs

CI/CD: Static websites

Products used in DevOps Toolchain:

Deployment

  • HashiCorp Terraform
  • AWS CodePipeline
  • AWS CodeBuild
  • AWS CodeDeploy
  • AWS ECR
  • Flyway

Config

  • AWS Systems Manager Parameter Store
  • Spring Cloud Config

Monitoring

  • Datadog
  • io
  • AWS CloudWatch
  • AWS CloudTrail

Testing

  • RunScope

Version control

  • AWS Systems Manager Parameter Store
  • Spring Cloud Config

Notifications and Collaboration

  • Slack

Advantages

The solution enables self-service of CI/CD pipelines where the customer DevOps team have full control over the IaC defined pipelines without direct access to Staging and Production environments.

The design supports separation of duty between the continuous improvements to microservices and the management of services and secrets in production. Encrypted secrets are kept within the realm and IAM of each account.

Improvements to the services and architecture are continuously discussed between Orange Business operations and Bokbasen DevOps through frequent dialog on slack as well as scheduled meetings.

The solution allows Bokbasen to integrate new functionality faster through continuous deployments, with greater degrees of reliability. The engagement with Orange Business has provided a sound footing for Bokbasen to undertake further migration activity towards AWS.

The basis part of the project has been realized so far, the “really cool” features will follow in the next years. The platform is already running 24/7 and is able to provide all systems with information. The next concrete step is to roll-out the “hardware” new passenger information platform.