The Best Credential Is No Credential: How I Moved Our Team to Keyless

After joining the team, I started reviewing how we used our cloud, and one question kept coming up: how were we handling our credentials, such as service account keys, database passwords, and LLM API keys? The answer revealed several problems.

Service account keys were stored as JSON files all over the place, often with overly broad permissions.
Our applications connected to the database over the public network using passwords.
We had piled up many unused LLM API keys, with nobody knowing who had created them or why.

Nothing bad had happened yet, but every one of those credentials was a door into our systems. A leaked service account key could let an attacker delete our infrastructure and take down our application; a leaked database password could expose sensitive data; a leaked LLM API key could run up our bill overnight. Storing them more carefully would only shrink the risk. I wanted to remove it, because the best credential is no credential.

How We Went Keyless

The best way to prevent credential leakage is to eliminate long-lived secrets entirely. We can easily do this on GCP, and on most other cloud platforms

Workload Identity Federation

Our CI/CD pipelines on GitLab need access to GCP: pushing container images to Artifact Registry, deploying to Cloud Run, running Terraform, and so on. The usual way to grant that access is to create a service account key, download the JSON file, and paste it into a CI variable. That key never expires on its own, and anyone who can read the CI variables, or any job that accidentally prints it, can walk away with it.

With Workload Identity Federation (WIF), we don’t store any key in GitLab at all. Each CI job receives an OIDC ID token signed by GitLab, which proves which project and branch the job is running on. GCP verifies that token, checks it against the conditions we set, and exchanges it for a short-lived access token for a service account. The token expires within an hour, so even if it leaks, it is useless soon after. Gitlab CI document - Configure OpenID Connect with GCP Workload Identity Federation

We also restrict which GitLab projects can impersonate the service account, so a job from some other repository can’t borrow our permissions.

resource "google_iam_workload_identity_pool" "gitlab" {
  workload_identity_pool_id = "gitlab-pool"
}

resource "google_iam_workload_identity_pool_provider" "gitlab" {
  workload_identity_pool_id          = google_iam_workload_identity_pool.gitlab.workload_identity_pool_id
  workload_identity_pool_provider_id = "gitlab-provider"

  attribute_mapping = {
    "google.subject"         = "assertion.sub"
    "attribute.project_path" = "assertion.project_path"
    "attribute.ref"          = "assertion.ref"
  }

  # only tokens from our own GitLab project are accepted
  attribute_condition = "assertion.project_path == \"hsingyingli/devops\""

  oidc {
    issuer_uri = "https://gitlab.com"
  }
}

resource "google_service_account_iam_member" "gitlab_ci" {
  service_account_id = google_service_account.ci.name
  role               = "roles/iam.workloadIdentityUser"
  member = "principalSet://iam.googleapis.com/${google_iam_workload_identity_pool.gitlab.name}/attribute.project_path/hsingyingli/devops"
}

deploy:
  id_tokens:
    GCP_ID_TOKEN:
      aud: https://iam.googleapis.com/projects/${GCP_PROJECT_NUMBER}/locations/global/workloadIdentityPools/gitlab-pool/providers/gitlab-provider
  script:
    - echo "$GCP_ID_TOKEN" > .ci_job_jwt_file
    - gcloud iam workload-identity-pools create-cred-config
        projects/${GCP_PROJECT_NUMBER}/locations/global/workloadIdentityPools/gitlab-pool/providers/gitlab-provider
        --service-account="${GCP_SERVICE_ACCOUNT}"
        --credential-source-file=.ci_job_jwt_file
        --output-file=.gcp_temp_cred.json
    - gcloud auth login --cred-file=.gcp_temp_cred.json
    - gcloud auth configure-docker asia-east1-docker.pkg.dev
    - docker push asia-east1-docker.pkg.dev/${GCP_PROJECT_ID}/my-repo/my-app:${CI_COMMIT_SHA}

Database Access: Cloud SQL Auth Proxy

Instead of connecting to Cloud SQL with a password over the public network, we now require every connection to go through the Cloud SQL Auth Proxy or a Cloud SQL connector, and only over a private network. The proxy authenticates with short-lived IAM credentials instead of a static password that never changes, so there is no password to leak. On top of that, we manage who can connect through IAM and service accounts instead of sharing database passwords.

resource "google_sql_database_instance" "main" {
  database_version    = "MYSQL_8_0"
  # ...

  settings {
    #...
    deletion_protection_enabled = true

    ip_configuration {
      ipv4_enabled    = false
      private_network = google_compute_network.vpc.id
      ssl_mode        = "ENCRYPTED_ONLY"
    }

    # only through cloud sql auth proxy or cloud sql connectors
    connector_enforcement = "REQUIRED"

    database_flags {
      name  = "cloudsql_iam_authentication"
      value = "on"
    }
  }
}

Switch From LLM API Key to Vertex

There have been plenty of stories of LLM API keys being stolen and running up huge bills overnight. Vertex AI avoids this risk entirely: instead of an API key, it authenticates through IAM, so access is controlled by service accounts just like the rest of our GCP resources.

genai.Client(
    vertexai=True,
    project=settings.gcp_project,
    location=settings.gcp_location,
    #...
)

Conclusion

Credential management is easy to overlook until something goes wrong, and by then it is usually too late. A single leaked credential can be enough to take down our infrastructure, expose our data, or run up the bill.

Our approach came down to two rules. First, wherever a service can authenticate with IAM, we use IAM. Second, where a workload runs outside GCP and can’t simply be a service account, like our GitLab CI pipelines, we use Workload Identity Federation to exchange its identity for a short-lived token. Then even if a token leaks, it expires within the hour, and the damage it can do is limited.