GCP Audit Logs

Set up the GCP Audit Logs connector.

In this guide, you'll configure the integration between Radiant and Google Cloud using Audit Logs and BigQuery. Audit Logs capture detailed records of activity across your Google Cloud projects, while BigQuery enables fast and flexible analysis of those logs for security monitoring and investigation.

Important note: GCP Audit Logs record all access events made by users and services within your environment. This may result in unexpected storage costs for both the logging service and BigQuery Storage. To understand the potential costs, we recommend reviewing the pricing guide for Logging. We also recommend assessing the pricing for heavily used resources in your environment, such as BigTable. It’s also a good practice to monitor your billing forecast and Logging Storage usage which can be found under Monitoring > Logs Storage.

Prerequisites

Before you begin, ensure that you have the following permissions:

Enable audit logs

As previously noted, some services may generate high volumes of logs, potentially increasing your billing costs. We recommend enabling logging for all services by following steps 4 and 5. If you later find that specific services are generating excessive logs, you can disable logging for them. To disable a specific log, follow step 6.

  1. Access the Google Cloud console.

  2. Select your Organization as the scope on the top part of the page.

  3. From the left side menu, navigate to IAM & Admin > Audit Logs.

  4. Scroll to the end of the page and select 200 as the number of rows per page. This will activate all logs at once:

  1. Click on the first checkbox to select all services. In the window that appears, under Permission Type, select Admin Read, Data Read and Data Write. Click Save.

  1. (Optional) To disable logs for selected services:

    • Search for and select the service that you want disable. For example, BigTable.

    • In the window that appears, under Permission Types, ensure that all the log types are unselected. Click Save.

Create a service account

If you have already created a service account for Security Command Center (SCC) connector, you might be able to reuse it. If that isn’t the case, select a project to which the service account will belong. The only constraint is that the service account needs to be in the same project as the BigQuery dataset that we will create.

  1. Go to IAM & Admin > Service Accounts.

  2. Click Create Service Account.

  1. Under Service account name, enter radiant-audit-logs-connector.

  2. Under Service account description, enter a descriptive name for easy identification later.

  1. Copy the email address generated and save it for later use.

  2. Click Create and Continue.

  3. In the Role section, add the following roles:

    • Log Viewer

    • BigQuery Admin

  1. Click Continue and then Done.

Create service account keys

  1. Click on the newly created service account to open its details.

  2. Click the Keys tab.

  3. Click Add Key and choose Create New Key.

  1. Select JSON and click Create.

  2. The JSON file will download automatically. Securely store it as it contains the service account key.

Create a BigQuery Dataset

  1. From the left side menu, navigate to BigQuery Console.

  2. In the left panel, expand the menu next to your project and click Create dataset.

  1. Under Dataset ID, enter radiant_connector.

  2. Under Default maximum table age, set it to 30 Days.

  1. Click Create Dataset.

Note: Double check the spelling of the Dataset ID as it’s needed to locate the dataset in a later step.

Create a log sink

Important note: For this step, ensure that you are in the organization scope again.

  1. Go to Logging > Log Router.

  2. Click Create sink.

  3. Under Sink name, enter radiant_audit_logs as a name.

  4. Under Sink description, enter a descriptive name for easy identification later.

  5. Click Next.

  1. Under Select sink service, select BigQuery.

  2. For Sink destination, select Use a BigQuery dataset in a project.

GCP will automatically fill the Sink destination as: bigquery.googleapis.com/projects/[PROJECT_ID]/datasets/[DATASET_ID]

  • Replace PROJECT_ID with the ID of the project where you created the dataset.

  • Replace the DATASET_ID with the value created earlier: radiant_connector

  1. Leave Use partitioned tables as unselected and click Next.

  2. Select Include logs ingested by this organization and all child resources. Leave the Build inclusion filter section empty and click Next.

  1. Leave the Build exclusion filter empty and click Create Sink.

Note: If you get the error Permission Denied, make sure you have the role Logging Admin at the Organization Level. This might happen even if you have the role Organization Admin.

Verify incoming Logs in BigQuery

  1. Go back to the BigQuery Console.

  2. Navigate to the dataset you created.

  3. Check for incoming logs to ensure the setup is working correctly.

PreviousFortinet Fortigate (syslog)NextGCP Security Command Center (SCC)

Last updated