Amazon S3

Connect Amazon S3 to WasteNot to import event files from your own bucket. S3 is useful when your team can export customer activity as files instead of giving WasteNot direct access to a database or application API.

WasteNot reads from your bucket with cross-account IAM role assumption. No long-lived AWS access keys are stored in WasteNot.

Prerequisites

• An Amazon S3 bucket containing the files WasteNot should import
• Permission to create or update an IAM role in the AWS account that owns the bucket
• The AWS region for the bucket, such as us-east-1
• A file layout with a separate prefix for each event type you want to import
• Event files in CSV format

File format

WasteNot imports event-based CSV files. Use a separate file or directory prefix for each event type, such as email opens, purchases, conversions, refunds, or support events.

Each CSV file must include a header row.

Each row should include these fields:

Example CSV:

id,email,phone,value,occurred_at,vendor_synced_at,customer_id
order_1001,camellia@example.com,+14155550123,129.95,2026-07-06 14:30:00,2026-07-06 14:45:00,cust_123
refund_1001,camellia@example.com,+14155550123,-29.95,2026-07-07 09:00:00,2026-07-07 09:05:00,cust_123

Recommended S3 layout

Create one prefix per event type. Each WasteNot stream maps one event type to one S3 prefix. For example:

s3://your-bucket/wastenot/orders/
s3://your-bucket/wastenot/refunds/
s3://your-bucket/wastenot/email_opens/

For the initial backfill, you can upload one or more historical CSV files to each prefix. For ongoing syncs, the easiest pattern is to upload new daily CSV files containing records inserted or updated since the previous run.

For example:

s3://your-bucket/wastenot/orders/backfill/orders_2020_2026.csv
s3://your-bucket/wastenot/orders/daily/orders_2026_07_06.csv
s3://your-bucket/wastenot/orders/daily/orders_2026_07_07.csv

Avoid repeatedly replacing one file with all historical event data if your team can produce incremental files. WasteNot can work with either pattern, but incremental files are easier to operate and debug.

Connecting S3

Step 1: Create the data source

1. Add a new Amazon S3 data source in WasteNot
2. Enter a Name
3. Enter the IAM Role ARN for the role WasteNot should assume, or create the role in AWS first and return to this form
4. Enter the Bucket Region
5. Optionally enter an External ID
6. Click Save

Step 2: Create the IAM role

On the S3 data source detail page, WasteNot shows the trust policy your AWS role should use. Copy that trust policy into the IAM role in your AWS account. It allows WasteNot's AWS role to assume your role and includes the correct external ID condition.

Then attach a read-only S3 policy scoped to the bucket and prefixes you plan to use in WasteNot streams. For example:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["s3:ListBucket"],
      "Resource": "arn:aws:s3:::your-bucket",
      "Condition": {
        "StringLike": {
          "s3:prefix": ["wastenot/*"]
        }
      }
    },
    {
      "Effect": "Allow",
      "Action": ["s3:GetObject"],
      "Resource": "arn:aws:s3:::your-bucket/wastenot/*"
    }
  ]
}

Replace your-bucket and wastenot/* with your actual bucket and stream prefixes.

Step 3: Add streams

After the data source is created, add one stream for each event type. The stream is where you choose the event type and S3 prefix; the data source only stores the shared AWS role and bucket region.

1. Open the Amazon S3 data source in WasteNot
2. Click New Stream
3. Choose the WasteNot event type, such as orders, refunds, or email opens
4. Enter the S3 URI for the prefix, such as s3://your-bucket/wastenot/orders/
5. Check Enabled and click Save

WasteNot will scan each enabled stream prefix and import matching files.

Frequently asked questions

Should I send one file or many files?

For the initial setup, one backfill file or a small set of backfill files is fine. For ongoing syncs, send new files containing records inserted or updated since the previous run.

Can each event type use a different file format?

No. S3 imports support CSV only for now.

What timezone should timestamps use?

Use UTC. The timestamp fields are naive timestamps without timezone data, so WasteNot treats them as UTC.

Does WasteNot need write access to my bucket?

No. The role only needs s3:ListBucket on the relevant bucket and s3:GetObject on the prefixes WasteNot imports.

My data source shows "Failed" status

Common causes include:

• The IAM role ARN was copied incorrectly
• The trust policy does not match the policy shown in WasteNot
• The external ID in AWS does not match the data source
• The bucket region in WasteNot is wrong

My stream shows "Failed" status

Common causes include:

• The S3 read policy does not include the bucket or prefix used by the stream
• The stream is mapped to the wrong event type
• The stream's S3 URI is not a valid prefix, such as s3://your-bucket/wastenot/orders/
• The bucket or prefix was renamed after the stream was created

My file shows "Failed" status

Common causes include:

• Files are not valid CSV files, or required fields are missing
• The CSV header does not match the expected field names
• A required timestamp field is blank or cannot be parsed
• A required id value is blank
• The file was uploaded outside the stream prefix

Check with your data team, or reach out to support@wastenot.io for help.