# For Data Engineers

This is the path for engineers who need programmatic access: SnowSQL, dbt, and a local development workflow. If you only need to read curated data in the browser, use **For Researchers & Analysts** instead.

## Prerequisites

- A Natera email address (`your_name@natera.com`)
- GitLab access to the `eng/cgdb` group on `gitlab.natera.com`
- AWS CLI configured with the appropriate profiles (`cgdb-sbx`, `cgdb-qa`, `cgdb-prod`)

## Step 1: Get provisioned in Snowflake

An admin adds your user to the `ndp-account-request` repo under `accounts/rwd/{env}/teams/rwd/users.json`. Once the MR merges, Terraform creates your Snowflake user and provisions the Okta SSO tile. See **How to Grant Access** for the full workflow.

## Step 2: Set up authentication for CLI access

**Programmatic Access Tokens (PATs)** are the simplest way to authenticate SnowSQL — no key generation or admin involvement. In Snowsight: your name → Settings → Authentication → Programmatic Access Tokens → Generate. Copy the token (you won’t see it again) and add it to `~/.snowsql/config`.

> **Key-pair auth**
>
> Key-pair authentication is for service accounts and CI/CD pipelines where tokens may expire. It requires an admin to attach the public key to your Snowflake user.

## Step 3: Install and configure SnowSQL

```bash
brew install --cask snowflake-snowsql
```

Add a connection profile per environment to `~/.snowsql/config`:

```ini
[connections.rwd_dev]
accountname = WA16250-NATERA_RWD_DEVELOPMENT
username = YOUR_EMAIL@NATERA.COM
password = <your-dev-PAT>
rolename = RWD_DATA_ENGINEER_ROLE
dbname = CLINICOGENOMICS
```

Always pass `-c` (connection), `-w` (warehouse — required), and `--noup` (suppress upgrade prompts):

```bash
snowsql -c rwd_dev -w RWD_DEVELOPMENT_QUERY_INTERACTIVE_WH --noup \
  -q "SELECT COUNT(*) FROM CLINICOGENOMICS.LIMS_PUB.CASEFILE_FILTERED;"
```

## Step 4: Install and configure dbt

```bash
uv tool install dbt-core --with dbt-snowflake --python 3.12
```

Configure `~/.dbt/profiles.yml` with `dev`, `qa`, and `prod` targets, then test the connection:

```bash
cd snowflake/dbt && dbt debug --target dev
```

## Step 5: Development workflow

1. Clone the `cgdb` repo from GitLab.
2. Create a worktree for your feature branch.
3. Make changes to dbt models in `snowflake/dbt/`.
4. Test locally against dev.
5. Create a Merge Request, get review, and merge to master.
6. CI/CD deploys to environments based on maturity tags.

```bash
git worktree add ../cgdb-CGDB-XXXX -b CGDB-XXXX-description origin/master
cd snowflake/dbt && dbt run --select my_model --target dev
```

> **Learn the build pipeline**
>
> For the full repository structure, what dbt manages versus what requires separate deployment, and maturity tags, see **dbt & Build Pipeline** under Engineering & Admin.

## MCP connections (Claude Code / Cursor)

If you use Claude Code or Cursor, Snowflake MCP servers can be configured in `~/.claude/mcp.json`. Default to the dev MCP server for all development; use preprod/prod only when explicitly targeting those environments.