# Configure Amazon S3 to forward logs to Radiant Security

In this guide, you will configure Amazon S3 (Amazon Simple Storage Service) to forward logs to Radiant Security. The S3 data connector should be used whenever a direct connector is not yet supported by Radiant Security.

{% hint style="info" %}
When configuring your data feed to deliver logs into S3, make sure the exported files match the expected **format and file extension** for each connector type. This ensures that we can correctly parse and forward the data to Radiant without additional configuration.
{% endhint %}

| Data feed                        | Expected format & extension          |
| -------------------------------- | ------------------------------------ |
| Akamai CDN                       | `.json`                              |
| Aurora Endpoint Security         | `.json`                              |
| Cloudflare WAF                   | `.log.gz`                            |
| GitHub Enterprise                | `.json.log.gz`                       |
| Imperva SecureSphere             | `.cef`                               |
| Imperva Cloud WAF (Incapsula)    | `.log`                               |
| Microsoft Defender For Endpoint  | `.json`                              |
| Palo Alto Firewall               | `.csv`                               |
| Suricata IDS                     | `.json`                              |
| Trend Micro TippingPoint IPS     | `.csv`                               |
| Twingate VPN                     | `.json`                              |
| Zeek Network Security Monitoring | `.json`                              |
| Custom Data Feed                 | any (uncompressed `.json` preferred) |

### Configure S3 and SNS

Review the following information regarding the S3 bucket configuration before proceeding with the setup of S3 and SNS:

* Our preferred region for the S3 bucket is `us-west-2`, but choosing this region is *not* obligatory
* You may use an existing S3 bucket, but only newly added files will be synchronized

Assuming you already have the bucket to which data is being sent, the first step is to configure the bucket to allow `GetObject` from our AWS Account ID (`AllowCrossAccountGetObject`).

1. Sign in to the AWS Management Console and open the Amazon S3 console.
2. In the **Buckets** page, click the **Permissions** tab on the bucket that you want to configure to forward data to Radiant Security.
3. For **Bucket policy**, click **Edit**.

<figure><img src="https://2439665791-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FPsFulb2ZOtSPcRSc2rXE%2Fuploads%2FegmgCNDJvcMExVbLE9Vq%2FConfigure_Amazon_S3_to_Forward_01.webp?alt=media&#x26;token=75fa2d59-152c-49b7-81e0-60f3a27694d7" alt=""><figcaption></figcaption></figure>

4. Copy and paste the following policy:

{% hint style="info" %}
If you are an E.U. tenant, replace `649384204969` in the statement below for `076657324990`
{% endhint %}

```
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "RSCollectLogs",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::649384204969:role/rs-connector-generic-aws-s3"
            },
            "Action": "s3:GetObject",
            "Resource": "<YOUR-BUCKET-ARN>/*"
        }
    ]
}
```

5. We highly recommend creating a new SNS topic specifically for this integration, rather than using a preexisting one. Note that a single SNS topic can be associated with multiple S3 buckets.&#x20;
   * On the navigation panel, click **Topics** and click **Create topic** to create an SNS topic.&#x20;
   * **Important:** In the **Details** section, for **Type**, select **Standard**. Amazon S3 only supports publishing to Standard SNS topics.
6. Once you’ve created the SNS topic, click **Edit**.

<figure><img src="https://2439665791-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FPsFulb2ZOtSPcRSc2rXE%2Fuploads%2F7S8PzJXOQqFpfnM6iBdH%2FConfigure_Amazon_S3_to_Forward_02.webp?alt=media&#x26;token=1cc3ac2f-30f6-4ba6-bca8-08afdf6d9adb" alt=""><figcaption></figcaption></figure>

7. Expand **Access policy.**

<figure><img src="https://2439665791-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FPsFulb2ZOtSPcRSc2rXE%2Fuploads%2FAA4OD7OBMeB344rXxPZW%2FConfigure_Amazon_S3_to_Forward_03.webp?alt=media&#x26;token=5aa28435-c835-4c36-bdd4-235634d329e2" alt=""><figcaption></figcaption></figure>

8. Copy and paste the following policies to allow your S3 bucket to publish to your SNS and to allow Radiant to subscribe to your SNS topic:

{% hint style="info" %}
If you are an E.U. tenant, replace `649384204969` in the statement below for `076657324990`
{% endhint %}

```
    {
      "Sid": "AllowS3Publish",
      "Effect": "Allow",
      "Principal": {
        "Service": "s3.amazonaws.com"
      },
      "Action": "SNS:Publish",
      "Resource": "<SNS-TOPIC-ARN>",
      "Condition": {
        "StringEquals": {
          "aws:SourceAccount": "<YOUR-AWS-ACCOUNT-ID>"
        },
        "ArnEquals": {
          "aws:SourceArn": "<S3-BUCKET-ARN>"
        }
      }
    },
    {
      "Sid": "RSSubscribe",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::649384204969:role/rs-connector-generic-aws-s3"
      },
      "Action": "SNS:Subscribe",
      "Resource": "<SNS-TOPIC-ARN>"
    }
```

### Add the data connector in Radiant Security

Add the Amazon Web Services S3 data connector in Radiant Security.

1. Log in to [Radiant Security](https://app.radiantsecurity.ai/).
2. From the navigation menu, click **Settings** > **Data Connectors** and click **+ Add Connector**.
3. Search for and select the **Amazon Web Services S3** option and then click **Data Feeds**.
4. Select the data feed(s) you want and then click **Credentials**.
5. Under **Credential Name**, give the credential an identifiable name (e.g. `AWS S3 Credentials`). If you already have the credential created, select it from the drop-down menu.
6. Under **Required Credentials**, enter the following:
   * **AWS Account ID**: The 12-digit number, that uniquely identifies your AWS account.
   * **AWS Region**: The region where your bucket resides (e.g. `us-west-2` or `eu-central-1`).
   * **SNS Topic Name**: The name of the SNS topic you created in step 5 of [Configure S3 and SNS](#configure-s3-and-sns)&#x20;
7. Click **Add Connector**.
8. Once the connector is successfully created, go to **Data Connectors**, locate the connector, and click **View Details**.
9. In the **Credentials** section, you will find an **Event name** for each data feed. Copy these names, as you will need them in the following steps.

### Configure the S3 events

To enable automated notifications when new data is added to your S3 bucket, you need to configure event notifications by following the steps below.

1. In the **Buckets** page, select the bucket that you want to enable events for.
2. Click the **Properties** tab.
3. Navigate to the **Event Notifications** section and click **Create event notification**.
4. In the **General configuration section**, configure the following:
   1. For **Event name**, enter the **Event name** first value that you copied in the last step of the [Add the data connector in Radiant Security](#add-the-data-connector-in-radiant-security) section.
   2. For **Prefix**, enter a string, such as `images/` to match the folder where that specific type of data is. You can use this to create a prefix filter so that you receive notifications only when files are added to a specific folder.
   3. In the **Event types** section, select **All object create events**. Leave all other options unselected.
   4. In the **Destinatio**n section, select **SNS topic** and search for the **Standard SNS** topic you had created.
   5. Click **Save changes**, and repeat this process for each one of your data feeds.

{% hint style="warning" %}
**Important note:** Forwarding data to Radiant Security under a different data type will cause that data to not be properly ingested. Be careful to separate the data types in the **Prefix** structure properly in case a same bucket hosts more than one type of data.
{% endhint %}


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://help.radiantsecurity.ai/radiant-connectors/data-connectors/configure-amazon-s3-to-forward-logs-to-radiant-security.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.