How to manage (monitor, log & secure) Cloud resources in my Domain

Technical additional documentation

Introduction

The objective of this article is to make an inventory of the various Flexible Engine services available to secure, monitor & manage logs of the Cloud ressources you will create inside your Domain. We will detail the relationships between them, the prior configuration to do (if necessary) and the activation for a given resource.

In a global way, Flexible Engine provides you with

  • A Supervision & Monitoring service: Cloud Eye (CES) is a multi-dimensional resource monitoring service. You can use Cloud Eye to monitor resources, set alarm rules, identify resource exceptions, and quickly respond to resource changes. More information here
  • A Logs management system :
    • A storage service : Log Tank Service (LTS) enables you to collect logs from hosts and cloud services for centralized management, and analyze large volumes of logs efficiently, securely, and in real time. LTS provides you with the insights for optimizing the availability and performance of cloud services and applications. More information here
    • Some collectors to collect various logs
      • CTS provides a cloud service resource operation log. This allows querying, auditing, reporting of the operations log as well as storing logs.  In addition, this feature records all logs triggered by API and console. More information here
      • VPC Flow logs help you monitor network traffic, analyze network attacks, and determine if security group and network ACL rules need to be modified. More information here
      • System & application logs via ICAgent: install an agent on the target Agency to be able to view logs on LTS
    • A Security service dedicated to ECS & BMS : Host Security Service (HSS) helps you identify and manage the assets on your servers, eliminate risks, and defend against intrusions and web page tampering. There are also advanced protection and security operations functions available to help you easily detect and handle threats. More d’information here
    • A notification service Simple Message Notification (SMN), enabling to send messages to various endpoints, such as phone numbers, and email addresses on any event generated by the services above. SMN Documentation is available here

 

To get the benefit of those services, you should

  • set the service (sometimes) to enable it
  • configure the cloud ressource to manage it with those services

Global configuration (project level)

These configuration should be done at the project level in your own Domain. Prior to any use, you should enable LTS, CTS, HSS respecting the following steps :

  • Create an ECS/BMS Agency to authorize the ECS agent to use LTS & CES & HSS

Then, the goal is to setup the log storage in LTS for short term retention, and OBS for long term, to collect any existing log collector

  • Create an obs-log bucket on the Object Storage Service
  • Create an LTS Log Group
    • One topic for lts-topic-icagent > log transfer to obs-log
    • One topic for lts-topic-flowlogs > log transfer to obs-log
  • Create a CTS Tracker for the prject, nothing more to do at the ressource level, see below)
    • Tracker Name = system > Trace Analysis Path = CTS/system-trace
    • To store logs in a bucket, there are two possibilities: either enable “Transfer Trace to OBS” on CTS or enable it at the LTS level (Log transfer, more information on helpcenter)

How to enable the services (Cloud resource level)

Cloud Eye Service (CES)

CES can propose some extended Server metrics if you run a monitoring agent (“telescope” agent) in each ECS/BMS you deploy. To use setup that, for any ECS/BMS, you need to :

  • Enable the agency (check above) in the ECS/BMS configuration
  • Install the telescope agent in the ECS/BMS. Check the helpcenter to setup the agent

Log Tank Service (LTS)

LTS can be used with :

  • CTS : enabling trace analysis on CTS is global, nothing to do at Cloud ressource levelc
  • VPC Flow Log : flow logs should be enable at NIC level, for any private IP of any ECS/BMS. To deploy VPC Flow Log, please follow the steps available here
  • System & application logs via ICAgent  : to collect such logs, you should run “ICAgent” in each ECS/BMS and the configure the log path
    • Enable the agency (check above) in the ECS/BMS configuration
    • Install the telescope agent in the ECS/BMS, check the helpcenter to setup the ICAgent
    • Configure the log path to collect logs in LTS, check Log ingestion in the helpcenter

Host Security Service (HSS)

To use HSS for any ECS/BMS, you need to :

  • Enable the agency  (check above) in the ECS configuration
  • Install the HSS agent : T
  • The steps to deploy HSS on your ECS/BMS are available on the helpcenter
  • Once the agent is installed, it is mandatory to configure an alarm. This can be done by choosing an SMN Topic to send the alarm notification via SMS, email, etc… An alarm notification will be sent in case of intrusion or other security issues
  • Enable the Server protection, according to one level of protection available, check helpcenter to choose