Identity Server 4 On Kubernetes Nginx Ingress

Have you ever tried deploying Identity Server 4¹ on a k8s (Kubernetes²) setup with Nginx³ ingress?

If you tried, I’m sure you’ve encountered some problems, as the current Nginx ingress is not properly configured for ASP.Net project or does not contain better optimization for Identity Server 4.

---

The first step towards getting somewhere is to decide you’re not going to stay where you are.

— J.P. Morgan.

Come on join me as we dive into the configurations!

Prerequisites

First of all, you must have a Kubernetes on your machine. Second, must have existing test bed project for Identity Server 4.

If you don’t have Kubernetes, perhaps you could try installing MicroK8s. The MicroK8s works on windows and MacOS.

So where do we start?

First, we modify the ingress ConfigMap configuration, and add the following lines:

proxy-buffer-size: "128k"  
proxy-buffers: "4 256k"  
proxy-busy-buffers-size: "256k"  
client-header-buffer-size: "64k"  
http2-max-field-size: "16k"  
http2-max-header-size: "128k"  
large-client-header-buffers: "8 64k"

This specific modification allows Identity Server 4 to send and receive large header data which is needed to store and sort out JWT (JSON Web Token) identifiers. You can check this sample setup on my test ingress config map YAML (Yet Another Markup Language):

kind: ConfigMap
apiVersion: v1
metadata:
  name: nginx-ingress-nginx-ingress
  namespace: default
  selfLink: /api/v1/namespaces/default/configmaps/nginx-ingress-nginx-ingress
  uid: 9fe8c06b-4f7c-4032-a938-505c308ed332
  resourceVersion: '10291469'
  creationTimestamp: '2020-09-18T12:46:50Z'
  labels:
    app.kubernetes.io/instance: nginx-ingress
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: nginx-ingress-nginx-ingress
    helm.sh/chart: nginx-ingress-0.6.1
  annotations:
    meta.helm.sh/release-name: nginx-ingress
    meta.helm.sh/release-namespace: default
data:
  client-header-buffer-size: 64k
  http2-max-field-size: 16k
  http2-max-header-size: 128k
  keepalive-timeout: '65'
  large-client-header-buffers: 8 64k
  proxy-buffer-size: 128k
  proxy-buffers: 4 256k
  proxy-busy-buffers-size: 256k
  proxy-http-version: '1.1'
  proxy-read-timeout: '150'
  sendfile: 'on'
  use-http2: 'false'

Next, thing we do is adjust our code to forward headers from and to ingress-app. The other method calls are also recommended by docs from Microsoft, you can check the setup here.

public void ConfigureServices(IServiceCollection services)  
{
    // ... code omitted ...
    // Needed for load balancer to forward headers
    services.Configure<ForwardedHeadersOptions>(options =>
    {
        options.ForwardedHeaders = ForwardedHeaders.XForwardedFor | ForwardedHeaders.XForwardedProto;
        options.RequireHeaderSymmetry = false;
        options.KnownNetworks.Clear();
        options.KnownProxies.Clear();
});

The docs specified the known networks / proxies are needed if you are hosting C# apps in a non-windows hosting environment.

After adding a forward headers configuration onto our ConfigureService method. We also need to add the forward headers middleware on the Configure method, which can also be found in Startup.cs file.

public void Configure(IApplicationBuilder app, IHostingEnvironment env)
{
    // ... code omitted ...
    app.UseForwardedHeaders();
    // ... code omitted ...
}

Then after that, restart the Nginx ingress and also your app to test whether everything is working fine. The next change is optional if you are using TLS.

If your ingress setup is TLS⁴ terminated. You also need to add this on your Configure method.

public void Configure(IApplicationBuilder app, IHostingEnvironment env)
{
    // ... code omitted ...
    app.Use(async (ctx, next) =>
    {
       ctx.Request.Scheme = "https";
       await next();
    });
    // ... code omitted ...
}

This specific custom middleware specifically converts all incoming calls to secured HTTP scheme. The TLS ingress specifically does is redirect the calls from your RS (Resource Server) to AS (Authorization Server) which is Identity Server 4 but TLS needs consistent HTTP secured scheme. If you look into your openid-configuration it will return http:// only endpoints and that is the problem, and that’s why we are modifying it internally using a custom middleware.

After all is done, restart the service and test every knick and knack.
That’s all guys!

Conclusion

It’s not just a simple clone image and deploy setup in k8s especially if you’re trying to deploy a c# app, sometimes you need to optimize some config in order for it to run smoothly /and or work well. Check the recommended deployment guide in Microsoft docs.

Let me know in the comments if you have questions or queries, you can also DM me directly.

Follow me for similar article, tips, and tricks ❤.

---

1. IdentityServer is an OpenID Connect provider – it implements the OpenID Connect and OAuth 2.0 protocols. ↩︎
2. Kubernetes is an open-source container–orchestration system for automating computer application deployment, scaling, and management. ↩︎
3. Nginx (pronounced “engine X”, /ˌɛndʒɪnˈɛks/ EN-jin-EKS), stylized as NGINX, nginx or NginX, is a web server that can also be used as a reverse proxy, load balancer, mail proxy and HTTP cache. ↩︎
4. Transport Layer Security (TLS), and its now-deprecated predecessor, Secure Sockets Layer (SSL), are cryptographic protocols designed to provide communications security over a computer network. Several versions of the protocols are widely used in applications such as web browsing, email, instant messaging, and voice over IP (VoIP). Websites can use TLS to secure all communications between their servers and web browsers. ↩︎