.NET Oxford January Meetup: NGINX with Ian Cooper

26 January 2019 - .NET , dotnetoxford , Meetups , NGINX

This month was our first .NET Oxford of 2019, and we were very pleased to welcome back Ian Cooper, who was this time talking about NGINX! For more details about the last time he joined us, see my blog post from March 2018 where he did a talk on 12 Factor Applications.

Whilst as usual I took copious notes during the talk for this blog post - frustratingly a certain note-taking app (who won't be named) decided to delete these notes and various others from my laptop! Apparently, I'm told - because of a cloud outage. I'm still not sure how a cloud outage removed notes from the same device they were created on though! Either way, it meant I was left without my notes when writing this blog post. Hopefully I can remember most of the talk though, and luckily there is also a slightly shorter version of Ian's talk online which helped refresh my memory. This video can be found here.

Obviously, whilst these blog posts are fairly detailed - they're still just summaries of the talks, and there's always a ton of information I can't capture in these write-ups. So I'd definitely recommend watching the above-mentioned video of Ian's talk, even if you do make it to the bottom of this post!

Intro Talk

This time is was Matt's turn to do the intro, and I also finally managed to take a photo that included the Everstack slide! :) Matt covered the usual house-keeping, then went through the news and price draws - doing a great job of looking far less of a muppet than I do when I'm doing the intro!

The published slides for the intro talk are available here, and the Reveal.js source code can be found on our Github account.

News

Just one news item this time - but it was a big one!...

GitHub: Announcing unlimited free private repos! Previously the free tier of GitHub only supported public repositories, and you had to have a paid subscription to create private repositories. With this announcement, you also get unlimited free private repos with the free tier! Which is great news - this will certainly save me a few pounds a month!

Ian Cooper - NGINX for .NET Developers

As mentioned above, this was Ian's second time at .NET Oxford. And like last time, his talk heavily featured the awesome world of Docker. Being a big fan of this style of architecture myself, and using it for most of my projects, I felt right at home. It's great hearing this kind of talk, helping more .NET developers feel comfortable with this way of working. Especially now that .NET Core is much more mainstream, and this is getting much more common.

ian

Ian started off introducing himself, and also highlighting a paragraph in his intro slide: "No smart guys: just the guys in the room". If I remember correctly he also made the same point at his last visit, stating that speakers stood up at the front aren't smarter than anyone else - the only difference is that they decided (and had the courage) to get up and speak infront of an audience to share their knowledge. Ian strongly encouraged everyone to give public speaking a try, and this is something I definitely agree with! Infact... always remember that we have lightning talk events were new speakers are most welcome - just get in touch and ask - even if you only want to start with just a 5-minute lightning talk, that's absolutely fine. You will find it surprisingly addictive!

Ian then spoke a bit about Linux. Whilst Windows containers are now becoming a thing - this container-style world still mostly revolves around Linux at the moment. Most .NET developers, prior to .NET Core, didn't really have many reasons to do anything with Linux. Ian points out that whilst this all might seem scary, and potentially put off .NET developers from considering using things like Docker and this style of architecture - it's not really that scary, it's just unfamiliar to a lot of .NET developers, and definitely worth the investment in learning.

He also mentioned an open-source project he works on called Brighter (and its sister project Darker). This is basically a CQRS framework, and there's a ton of information on the website about the project.

Ian's demos were using the same example GreetingsApp project as he used in the 12 Factor App talk he did at .NET Oxford last year. Note that when looking at that GitHub project, make sure you choose a relevant branch rather than the default master branch. He described the GreetingsApp as "hello world as a webserver". It's basically a CRUD webapi for 'Greetings'. The idea being that it's a very simple demo project that people can quickly understand, so Ian can focus on the topic he's trying to teach. He used this demo app for the various examples throughout the talk, switching to the relevant branch for that particular demo. He also showed that it uses his Brighter project (mentioned earlier) and gave a few examples of how this works.

He showed a docker-compose file with his greetings app web service, and also a mysql database. He also demoed making HTTP requests against his GreetingsApp with the VSCode RestClient extension, which is a fantastic extension that lets your write HTTP requests in text files, submit them, and view the response data directly in VSCode.

After showing a ton of stuff that wasn't NGINX ;), he then showed an example "nginx.conf" file, as well as the Dockerfile that copies this config file into his NGINX image. This basically means that when he runs the Greetings App mentioned above by running docker-compose up, all the services, eg. the app, database, and this NGINX image (with config file), will all be spun up together. All of this is specified in source-controlled configuration files, so anyone else can pull down the repository, and simply run docker-compose up, and it should just work. The only prerequisite being Docker.

Kestrel

One of the first topics he dived into was Kestrel, which is a cross-platform web-server for ASP.NET Core. In fact, it's the default embedded web-server when you run an ASP.NET Core app.

He went quite in-depth explaining how Kestrel came about and how it uses libuv to manage the number of requests it handles (although this has now changed to managed sockets), and also showed a chart showing how it compares to other web servers performance-wise (see slide 14 in his slides).

He explained that whilst Kestrel is a web-server, and you could use it as your main entry point - it lacks lack some of the functionality that you may need - eg. compression, proxy-cache, url-rewriting, mature TLS, etc. This is where a reverse-proxy like NGINX comes in. The Kestrel webpage itself even has a section on when and why you'd want to put a reverse proxy such as NGINX in-front of it.

IIS Architecture Comparison

Ian spent quite a bit of time comparing the typical IIS-style architecture that a lot of us .NET devs are used to with the Docker/container style architecture (see slide 19 in his slides). He went quite into detail with explaining the differences. IIS does a lot of of different things, and arguably, it actually does it quite well - but, it's a bit of a monolith, and is also Windows only. Whereas the Docker-style architecture is much more componentised and you can pick the right tool for a specific job - NGINX being one of them.

I should point out that Ian never said that one was better than the other - it was just a comparison. He also explained that it wouldn't make sense to mix the two - as IIS is an application container, and Docker is an application container - so there'd be no point putting one inside the other.

NGINX Configuration

At the heart of NGINX is the nginx.conf configuration file. This should be source-controlled, and allows you to control all the various things that NGINX provides. Including how different URL routes are processed, and what NGINX should do with those requests.

One of the core blocks in this config file is the http block. Below is a snippet from one of Ian's examples...

http {
  sendfile on;
  tcp_nopush on;
  tcp_nodelay on;

  server {
    listen 80;
    server_name www.example.com;

    location / {
      try_files $uri $uri/ @kestrel;
    }

    location @kestrel {
      proxy_pass         http://greetingsweb:5000;
      proxy_set_header   Host $host;
      proxy_set_header   X-Real-IP $remote_addr;
      proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header   X-Forwarded-Host $server_name;
    }
  }
}

The server block has location directives which specify how NGINX should handle a request for a particular URI. In the above example, you can see that a request coming in for the default route (/), it'll first check if there's a physical file that matches the path - if not, it'll pass over to the location @kestrel directive.

A lot of this is fairly intuitive. A core keyword to note is proxy_pass. This tells NGINX to forward on the request to something else. In this case, it's the Greetings webapp, which is the webapp service running from the docker-compose file mentioned above.

You can see that headers are also begin added to the request. You can also see that NGINX is both handing static content from disk, and also forwarding traffic using this proxy_pass keyword.

Outside of this location block, there are also plenty of other configuration settings, which Ian touched upon - for example, allowing you to configure the number of CPU cores that NGINX runs on.

Reverse Proxy and Caching

Ian explained that NGINX is a reverse proxy, and that this means that it sits on the server next to your webapp rather than on the client-side like a forward proxy (eg. a proxy your users' IT department set up). He covered quite a bit on different types of caching, comparing the different places where you might cache response data, and the pros and cons of each. For example, caching in the browser is as close to the end-user as you can get (so very responsive), but isn't shared with anyone else. However, caching close to the server (eg. in NGINX), is shared between all our users, so dramatically reduces hits on your application, but is obviously further away from the user. If your server is on the other side of the planet from the end-user, caching at the server isn't going to help with that round the world latency.

He also covered different types of browser headers for caching - for example, the max-age, expires, last-modified, cache-control headers. I won't go into all this here, as this blog post is getting long enough already - but there's a good article about http caching headers that can be found here if you want to learn more about this. The point being, that you can set these in your webapp, and like the browser, NGINX can also decide when to cache based on these response headers.

He then jumped back to another nginx.conf example showing how to configure NGINX's proxy cache, and spoke quite a bit how all this works, then moved onto a demo. Unfortunately the demo-gods came to play with what looked like a build issue, so he couldn't properly demo this caching in his Greetings App. Fortunately though, as mentioned earlier, his NDC talk is online with a working demo! Here's a link to the correct place in the video.

Load Balancing

Whilst you can put a single webapp behind an NGINX instance, you also have the option to use NGINX as a load balancer. Ian showed an example of the upstream directive in the config file, which looks something like this...

upstream backend {
    server web1;
    server web2;
    server web3;
}
server {
    listen 80;
    location / {
        proxy_pass http://backend;
    }
}

The above example is fairly straightforward - there's a list of target servers in the upstream directive, and we're using proxy_pass to point to the upstream block.

Ian's switched back to his demo to show a working example. Instead of the above, he just used a single server in the upstream block: server web:5000;, and demoed this using docker-compose up --scale web=3, so he had three instances of the webapp running.

He also showed another variation which used a resolver in NGINX, which I believe he only needed because he was using the old Docker Toolbox. Basically it means that NGINX will spot when Docker adds new containers.

SSL

Ian unfortunately ran out of time at this point, so he only touched upon SSL briefly. His example nginx.conf with the ssl directives can be found here, and you can see that it's pretty simple - it just references the certificate on disk.

URL Rewriting

Another topic which he explained briefly, but didn't have time to go into fully, was URL rewriting, allowing you to change the URL based on specified regular-expression based rules. This is useful for a number of reasons - eg. perhaps your URLs have changed on your website, but to avoid page not found errors with old links on the web, you want NGINX to do a 301 redirect to the new URL.

Wrapup and Links

Ian wrapped up by mentioning a few more things that NGINX does, but he didn't have time to cover in this talk. For example, health checks, redis and memcached support, and monitoring.

A massive thank you again to Ian for joining us for a second talk!

Below are some useful links to his slides, code, etc...

Ian's Slides
Demo code on Github (remember to choose the relevant branch rather than 'master')
Recent recording of the same talk at NDC
RestClient
Blog post from Ian's last .NET Oxford visit

Prize Draw Winners

During the intro talk, we also had our prize-draw giveaway using my usual WPF Prize Draw app. A massive congratulation to the winners, and a massive thank you to our awesome prize draw sponsors ...

Jetbrains

Congratulations to Redner for winning a year-long Jetbrains product licence! He very wisely choose the combined Resharper Ultimate + Rider pack!

Manning Books

Congratulations to Mark Osborn for winning a Manning ebook! The winner has the choice of any of the awesome Manning ebooks from their website.

Remember that we have our special Manning coupon code (ug367) which gives all of our members a 36% discount on any of their e-books! They've also asked me to share a link to some of their new courses for their LiveVideo system.

Oz-Code

Congratulations to James World for winning the Oz-Code licence!

For those that don't know, Oz-Code is a Visual Studio extension that puts your debugger on steroids. Take a look at their website for videos of their features. If you haven't checked it out, then definitely download the trial and have a play. All our member get a free 3 month trial licence (see below) or 50% off a full licence! To claim, you can visit this link to pick up your licence!

prizesponsors

Devtips

This time was a bit of a disappointment. Quite a few people in the break mentioned that they'd do one - but when it came down to it, there was just myself and Jake...

Dan Clarke: Using NGINX to get around http://localhost:XXXX

"Given the topic of the evening, I thought it worth throwing in an NGINX devtip! I use NGINX locally to avoid having to use URLs like http://localhost:5000 for local development. I've just written a blog with full details which can be found here."

Jake Peek: JS Date parsing in Safari Gotcha

"The Date class in JS only accepts certain patterns in certain browsers. Most notably, in Safari if you try and construct a date object like this: new Date("2019-01-01"), then you'll get an 'invalid date' returned. It's a particularly annoying 'gotcha' because a lot of databases store dates this way. It does however seem to work in most other browsers. You can get around it by swapping out '-' for '/'. So the same date constructed like this new Date("2019/01/01") works just fine."

Pub

After the meetup, as usual, a few of the regulars went down to the pub to continue the geek conversations. We were also joined by a couple of new members - Jake and Ben (left in the picture below), who work for a company called Jetfruit, and had recently reached out to me because they're also planning on starting their own meetup in Aylesbury about serverless technologies called Jet Set & Code. And at the time of writing, they've now had their first meetup, which I believe went really well! If you're near to Aylesbury, which isn't that far at all from Oxford, then definitely check their new meetup out!

pub

Upcoming Meetups

Below are our upcoming meetups. We'll be announcing more meetups for next year shortly, and if you subscribe to the meetup group, you'll get email notifications as they are announced.

12th February 2019: .NET Internals - Chris Bacon

In February, we'll be delving a bit deeper under the hood into .NET with Chris Bacon talking about .NET Internals.

12th March 2019: Functional C# - Simon Painter

Our March meetup will be with Simon Painter talking about writing Functional C#.

April 2019: Uncle Bob!

We're very excited to announce that in April, we'll be joined by Uncle Bob himself! A topic hasn't yet been decided - but we'll be announcing this on Meetup very shortly. We're going to aim for the larger St Aldates venue next door (where we had the Jon Skeet event) for this one.

May 2019: Lightning Talks

We're thinking of another lightning talk event for May. If you're interested in doing a 5, 10, 15, or 20 minute talk - then let us know!

Please retweet if you enjoyed this post ...

Blogged: ".NET Oxford January Meetup: NGINX with Ian Cooper" https://t.co/RSB68ImX4V @dotnetoxford @ICooper @mnield @CorriculoRec
— Dan Clarke (@dracan) January 26, 2019