BigQuery
Connect Google BigQuery to WasteNot to pull data directly from your warehouse. Write simple queries to define exactly what data you need for building audiences.
Setting up BigQuery requires help from someone on your engineering or data team. Share this guide with them to get started.
Prerequisites
- A Google Cloud project with BigQuery enabled
- A service account with access to the data you want to query
- Someone with access to your Google Cloud console or CLI
Connecting BigQuery
Setup has two steps: creating the data source in WasteNot, then linking it to your Google Cloud project.
Step 1: Create the data source
- Add a new BigQuery data source in WasteNot
- Enter a Name
- Click Save
Step 2: Link to your Google Cloud project
After creating the data source, go to its detail page. You will see a form and setup instructions.
Your engineering team will need to fill in:
- Project Id — Your Google Cloud project ID
- Workload Identity Pool Provider Id — Created during the setup process below
- Service Account Email — e.g.,
my-sa@my-project.iam.gserviceaccount.com
The page includes step-by-step setup instructions in three formats:
- ClickOps — Walk through it in the Google Cloud console
- gcloud — Copy and run a script in the terminal
- Terraform — Add a config block to your infrastructure code
Once the Google Cloud side is set up and the fields are filled in, click Connect.
WasteNot connects to BigQuery without storing any passwords or keys. If your team needs help, reach out to support@wastenot.io.
Custom streams
Once connected, you can create streams to define what data WasteNot pulls from BigQuery. Each stream is a SQL query that returns rows matching WasteNot's expected format.
See the Writing Custom Queries guide for full details on how to structure your queries, required columns, and event types.
- Go to your BigQuery data source detail page
- Click New Stream
- Give the stream a name, select an event type, and enter your SQL query
- Save the stream
WasteNot will run the query on a schedule and sync the results automatically.
Frequently asked questions
Who on my team should set this up?
Anyone with access to your Google Cloud project and permission to create service accounts. This is typically someone on your engineering or data team.
Can I query data from multiple datasets?
Yes, as long as the service account has access to them.
Something went wrong during setup
If you get stuck, reach out to support@wastenot.io — we can help walk your team through it.