Confluent Log Monitoring

Project Creation

1. Go to “Settings”->“System Settings”. Click on “Add New Project”.
2. Select “Custom” from the list and click on “Create Project” on the next page.
3. This is where you will start configuring the project.
   1. Instance Type – Private Cloud
   2. Data Type – Log
   3. Agent Type – Live Streaming
      (See Figure 1)
4. Click on “Next” and you will be taken to the next page.
5. Then on the next page, you can type in the “Project Name” and “System Name” and click on Register. You can also define detection keywords and Incident labels.
   1. 1. Detection keywords/regular expressions are used to detect which log entries will produce allowlist alerts.
      2. Incident labels/regex are used to identify which log entries indicate incidents.
         (See Figure 2)
   2. Once a project is successfully created, you will see a message like below.
      (See Figure 3)
   3. Once a project is successfully created, you need to set-up an agent.

Agent Setup

You would need a server where you can run this agent.

• For container:
  1. Access the image
  2. Follow the steps from the overview section to use the agent

• For GitHub:
  1. Download the agent from the link on the server.
  2. Follow the steps from the link to install the agent on the server.

Once you have an agent running, you will be able to see the data streaming into the InsightFinder project. To check,

1. Go to Analysis->Log/Trace Analysis.
2. Select your project and date. Click on the Refresh button on the right top. You will get the calendar view of log data.
   (See Figure 4)