AWS CloudWatch Integration

InsightFinder can source the data from Amazon Web Services (AWS) Cloud Watch and correlate it with other data to generate anomalies and root causes. Below documentation is a walk through of how to configure AWS CloudWatch and InsightFinder integration.

Project Configuration

Information needed: AWS Access Key ID and Secret Access Key for authentication.

Project Creation

  1. Go to “Settings”->“System Settings”. Click on “Add New Project”.
  2. Select “AWS CloudWatch” from the list and click on “Create Project” on the next page.
  3. This is where you will start configuring the project.
    1. Select “Instance Type” based on which AWS instance you want to monitor.
    2. Select “Region” based on where your AWS instance is located.
    3. Provide IAM Access Key ID, Secret Access Key.
      Permissions required: “ec2:DescribeInstances”, “logs:DescribeLogGroups”, ”cloudwatch:GetMetricData”, ”logs:DescribeLogStreams”, “servicediscovery:ListNamespaces”, ”cloudwatch:GetMetricStatistics”, “cloudwatch:ListMetrics”, ”ec2:DescribeSecurityGroups”, ”cloudwatch:ListMetricStreams”, “autoscaling:DescribeAutoScalingGroups”, ”rds:DescribeDBInstances”, ”logs:GetLogEvents”, “rds:DescribeDBProxies”, ”cloudwatch:GetMetricStream”, ”ec2:DescribeInstanceStatus”
    4. Select Metric or Log based on what you want to source into InsightFinder.
      (See Figure 1 and 2)
  4. Click on “Verify” and you will be taken to the next page if verification is successful.
  5. Select “Instance Type” from the dropdown that you want to configure for your analytics in the InsightFinder. Similarly select “Metrics” for those instances. If you want to configure “KPI Metrics” , then select those also. You can leave the rest as default or configure as needed.
    (See Figure 3, 4, and 5)
  6. Default sampling interval is 5 Min. Minimum could be 1 min. You can choose your sampling interval.
  7. Then on the next page, you can type in the “Project Name” and “System Name” and click on Register.
    (See Figure 6)
  8. At this stage for a log project, you will have an option to provide Detection or Incident Keywords which will be used by the system for detecting specific types of log events or creating incidents from logs. Keywords in the image are just examples.
    (See Figure 7)
  9. Once a project is successfully created, you will see a message like below.
    (See Figure 8)
  10. You should start seeing data in the Metric Analysis page as below.
    (See Figure 9)
  11. Similarly, you can configure other AWS instances for metrics and logs into InsightFinder.

Figures

Click on an image to expand.