Snippets demonstrating a simple way to embed/associate audit logged metadata to GCP API calls.

Or in other words, whenever you make any GCP API call like GCS, PubSub, you can automatically attach metadata to the call which will show up in audit logs. One usecase is to attach information about why the request is being made. For example, if you run a SaaS service that uses GCP APIs and you make calls to services on behalf of your customers, you can attach information to that call indicating who it was made for.

Also included in this repo is the basic ‘helloworld’ for OpenCensus. With this, you can also emit a metric/counter to any exporter OpenCensus supports that indicates who the call is for. Basically, you can have a metric that shows requests/second or any other metric per customer. …


Simple gRPC client/server for GCP API Gateway and Cloud Run…now with with authentication and authorization!

Basically you deploy a gRPC server on Cloud Run, then an API gateway which brokers requests. The gateway is protected by JWT authentication while the Cloud Run backend will only allow requests originating from the gateway’s service account identity

client -> (gRPC+auth) -> APIGateway -> (gRPC+auth) -> Cloud Run API Server

You can find the source here

other references:

Setup

Set environment variables

export PROJECT_ID=`gcloud config get-value core/project`

export PROJECT_NUMBER=`gcloud projects describe $PROJECT_ID…


A simple tutorial on how to setup envoy and istio such that per method statistics are emitted to prometheus+grafana.

There are several techniques to emitting and collecting gRPC stats shown here:

  • client-> envoy -> server: Envoy will surface prometheus stats endpoint
  • client-> istio(envoy) -> server: Istio (envoy) will again surface a prometheus endpoint
  • client (native prometheus gRPC): Use gRPC built in prometheus stats endpoint "github.com/grpc-ecosystem/go-grpc-prometheus"
  • client (opencensus prometheus): Use Opencensus's Prometheus exporter for gRPC "contrib.go.opencensus.io/exporter/prometheus"
  • client-> kubernetes ingress/service -> server: Deploy a GKE gRPC service and allow a prometheus server to scrape each gRPC server eposing "github.com/grpc-ecosystem/go-grpc-prometheus"

Both istio and envoy have robust metric monitoring capability but the default granularity for these are at the service level. If you needed resolution at the method you would need to some further configuration settings if using envoy or…


A really basic implementation of envoy External Processing Filter. This capability allows you to define an external gRPC server which can selectively process headers and payload/body of requests (see External Processing Filter PRD. Basically, your own unrestricted filter!

          ext_proc   (redact header from client to upstream)
^
|
client -> envoy -> upstream

NOTE, this filter is really early and has a lot of features to implement!

You can find the full source here:

All we will demonstrate in this repo is the most basic functionality: simply remove a specific heder sent by the client. I know, there are countless easier other ways to do this with envoy but just as a demonstration of writing the external gRPC server that this functionality uses. …


Procedure and referenced library that will exchange an arbitrary OIDC id_token for a GCP credential.

You can use the GCP credential then to access any service the mapped principal has GCP IAM permissions on.

The referenced library github.com/salrashid123/oauth2/google surfaces an the mapped credential as an oauth2.TokenSource for use in any GCP cloud library.

If the underlying credentials expire, this TokenSource will NOT automatically renew itself (thats out of scope since its an arbitrary source)

This repo is the second part that explores how to use the workload identity federation capability of GCP which allows for external principals (AWS,Azure or arbitrary OIDC provider) to map to a GCP credential. …


Sample procedure and referenced library that will exchange a long term or short term AWS credential for a GCP credential.

You can use the GCP credential then to access any service the mapped principal has GCP IAM permissions on.

The referenced library github.com/salrashid123/oauth2/google surfaces an the mapped credential as an oauth2.TokenSource for use in any GCP cloud library.

If the underlying credentials expire, the TokenSource will automatically renew itself, hands free.

This repo is the first part that explores how to use the workload identity federation capability of GCP which allows for external principals (AWS,Azure or arbitrary OIDC provider) to map to a GCP credential. …


This is a very basic STS (Secure Token Service) deployed on Cloud Run which exchanges a one access_token for another....basically, a token broker described here:

The STS server described here is not a reference implementation..just use this as a helloworld tutorial

This particular STS server exchanges one static access token for another. It will exchange

  • iamtheeggman for iamthewalrus (right, thats it..)

You can use an http client curl to see the exchange directly and then use a new gRPC client which utilizes its own gRPC STS Credential object:

This is not an officially supported Google…


Deterministic container images for gRPC+golang bazel.

The following sample will build a golang gRPC client/server and then embed the binaries into container images.

These images are will have a consistent image hash no matter where it is built.

For reference, see:

To run this sample, you will need bazel installed

Build Image

Note, the bazel base image specifies the image+hash so your’e starting off from a known state:

  • WORKSPACE

Check Image

Inspect the image thats generated…these wil be the same no matter where you generate the images

(optional) gRPC Client/Server

(why not?…you built it already, give it a…


Just a simple C# application that calculates the closest driver to a rider using Homomorphic Encryption.

This sample uses C# Microsoft SEAL Library to perform the distance calculation under encryption:

  1. On startup a Rider is assigned a random (x1, y1) coordinate in a 1000x1000 grid
  2. Rider generates homomorphic publickey, secretkey: pk, sk
  3. Rider encrypts the xy coordinates using pk: x1->x1', y1->y1'
  4. Rider sends encrypted x1', y1' and pk to server
  5. Server sends pk to 50 drivers each in a in random position in the grid
  6. Each driver encrypts their respective location x2,y2 using pk -> x2',y2'
  7. Each driver each sends x2', y2' to the…


Basic tutorial to synchronize contents of two Object Storage buckets using Rclone on Google Cloud.

The specific implementation here uses GCS Buckets as the source and destination but it would be relatively simple extension to other backend storage providers. For example, an AWS S3 configuration as source bucket is shown in the Appendix section.

This repository is NOT supported by Google.

This utility has not been tested at scale. Since the synchronization iterates files, it will not scale with very large numbers of objects or where each file transfer exceeds the default timeout of Cloud Run (15mins). However, there are techniques you can use to shard the iteration using Cloud Tasks across various Cloud Run instances. …

salmaan rashid

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store