C'est mirifique!

Nginx-Ingress Reverse Proxy in Front of an Object Storage for inflexible WebGL and stubborn CORS

Rationale: in an Unity App generating WebGL with remote-assets loading where we have very little control on the generated code, I’m limited to one solution and it’s to comply with CORS: hosting assets in the same domain as the app is running in.
In this context it seems indeed impossible to set correctly the crossorigin attribute in the generate WebGL code. If you think otherwise please teach me in the comments.

TLDR; It works fine of course, but don’t waste your time deploying your own reverse proxy if you want to keep the benefit of a CDN and if performances is a key in your deployment.

This wouldn’t be an issue if the current provider, DigitalOcean, would allowing me to add custom hostname to its Object Storage offer (Spaces). It actually allows it but only for domain managed by DigitalOcean itself. Which is nonsense to anyone able to add a DNS entry (anyone with two fingers and a tong?) and no-go for us as our DNS is hosted by CloudFlare and acts as a good protection for the infrastructure.

CloudFlare is a reverse proxy (and more) and hide our origin servers from the wild wild web. I would have expect this provider to allow me to proxy our origin server to any FQDN. For instance front.org.tld to forward requests and return content from cdn.location.digitaloceanspaces.com. This is unfortunately not available for the free tier and it seems available to the business plan under the name CNAME Setup, but I don’t agree with the price for such a “simple” feature. We are already paying with our data and it seems that concurrent are offering this feature to free tiers.

I excluded the idea to “write” a reverse proxy in Javascript and use CloudFlare workers because… well it’s nonsense shitty tech. The net is not a trash and needs no more bullshit solutions.

Our services are running in a Kubernetes cluster so if I tolerate the performance trade off to run a reverse proxy in my own rented infrastructure, this solution is “free”, relatively clean and allow us to get forward with our project.

There is drawbacks. Many. For anticipated heads-up and warnings, see the end of this post.

With the bellow service and ingress settings, requests sent to OBJECT-STORAGE-RP.ORG-DOMAIN.TLD (any FQDN) will return content from OBJECT-STORAGE-HOSTNAME (Object Storage on DigitalOcean Spaces, Amazon S3, Google Cloud Storage, etc.).

---
apiVersion: v1
kind: Service
metadata:TODO
  namespace: APP-NAMESPACES
  name: OBJECT-STORAGE-RP
  labels:
    app: OBJECT-STORAGE-RP
spec:
  externalName: OBJECT-STORAGE-HOSTNAME
  type: ExternalName
  selector:
    app: OBJECT-STORAGE-RP

---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  namespace: APP-NAMESPACES
  annotations:
    kubernetes.io/ingress.class: "nginx"
    cert-manager.io/cluster-issuer: ORG-SSL-ISSUER
    nginx.ingress.kubernetes.io/backend-protocol: https
    nginx.ingress.kubernetes.io/upstream-vhost: "OBJECT-STORAGE-HOSTNAME"
    nginx.ingress.kubernetes.io/server-snippet: |
      proxy_ssl_name OBJECT-STORAGE-HOSTNAME;
      proxy_ssl_server_name on;
  name: OBJECT-STORAGE-RP-ingress
  labels:
    app: OBJECT-STORAGE-RP

spec:
  tls:
  - hosts:
    - OBJECT-STORAGE-RP.ORG-DOMAIN.TLD
    secretName: OBJECT-STORAGE-RP-ORG-TLS
  rules:
  - host: OBJECT-STORAGE-RP.ORG-DOMAIN.TLD
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: OBJECT-STORAGE-RP
            port:
              number: 443

Where variables are:

  • OBJECT-STORAGE-RP = a name to define this reverse proxy
  • OBJECT-STORAGE-HOSTNAME = source host, where assets are hosted (Spaces in my case)
  • ORG-SSL-ISSUER = when using SSL, might be letsencrypt (certbot) and this is cluster-dependent
  • APP-NAMESPACES = a meaningful or random namespace
  • OBJECT-STORAGE-RP.ORG-DOMAIN.TLD = the reverse proxy FQDN, might completely differ from OBJECT-STORAGE-RP, or not

On one hand there is notable performances trade offs: most object storage provider offer a CDN option to boost assets access in client’s browsers and this feature won’t be a viable option.
If you use signed access it won’t be possible to add a caching CDN in front of this reverse proxy. This instance can still be hidden by CloudFlare for instance, but with no performances improvement as the caching won’t work.

Doing so I’m wasting resources because my queries are going through many gateways (proxies) and encrypt/decrypt queries for nothing more than renaming. And I’m paying (a bit) for it.

On the other hand there is advantages. Doing so is also blurring lines, which is a security improvement (and even a business if you look at CloudFront and other Cloud-Behemoth).

It’s working, settings are flexible and this means that we can continue to work without CORS issues.

However this is not a solution in itself. The right way would be either of the following:

  • being able to setup our own hostname (FQDN) directly for the bucket of our Object Storage provider.
  • setting up the Cloud reverse proxy directly to point to the right origin server. CloudFlare in our case.

Resources:

Posted

in

by

TO

Tags: