Google Cloud Cost

Summary – InsightFinder can stream the cloud cost data from google cloud and generate anomalies to automatically alert the team of any unwanted or unidentified activities in the account from hackers or over utilization and help reduce your cloud bills.

Project Creation :-

1. Go to “Settings”->“System Settings”. Click on “Add New Project”.
2. Select “Google Cloud Cost” from the list and click on “Create Project” on the next page.
3. This is where you will start configuring the project.
   1. Project Id, Private key Id, Private key, Client email, Client id – Use your gcp authentication key information here. Please Note – Private Key starts with 5 ‘-’ and ends with ‘\n’.
   2. Another way is for users to directly upload json key file.

   

4. Click on “Verify” and you will be taken to the next page, if verification is successful.
5. Select the billing table from the list and specify any historical period. (Refer next section to understand the gcp billing table setup). You can also specify the sampling interval, as well as the collection frequency (how often data is pulled from BigQuery).
   
   
6. Then on the next page, you can type in the “Project Name” and “System Name” and click on Register.
7. Once a project is successfully created, you will see a message like below.
   

Once a project is successfully created, you will see data in the metric chart like below.

GCP billing table setup :-
First, be sure you have either the Billing Account Costs Manager role or the Billing Account Administrator role on the billing account for which you want to fetch billing data. Ensure that the service account you will use has the BigQuery data viewer role for this table.

1. Use an existing project or create a new project for billing dataset at https://console.cloud.google.com/projectselector2/home/dashboard.
2. Link the project to the billing account from which you want to fetch data. https://cloud.google.com/billing/docs/how-to/modify-project#confirm_billing_is_enabled_on_a_project.
3. Add the BigQuery User role for the project you are using.
4. Create a dataset in BigQuery under the project you plan to use by going to the console at https://console.cloud.google.com/bigquery and in the left menu click “Create dataset”
5. Give your dataset a name and fill out other fields as desired. If you are going to use Detailed usage cost data (see below), you will need to choose either the mult-region US location or the multi-region EU location. Do not enable table expiration. In the advanced settings, leave Encryption as “Google-managed encryption key”.
6. Once you have created the dataset, go to the Billing export page at https://console.cloud.google.com/billing/export and select “EDIT SETTINGS” under that export. Choose the project and dataset you chose/created in the previous steps and click “SAVE.” Enable the BigQuery API if you are prompted to do so.Figure –
7. You should be able to see the table in the BigQuery console under the dataset you created earlier and the table that will appear in the InsightFinder configuration list.

Warning: Use of BigQuery can result in Google Cloud costs. However, the cost is either 0 or a small amount depending on the existing usage of BigQuery.