Coding Fun – NGINX reverse proxy

YANRPA – Yet Another Nginx Reverse Proxy Article.  Seriously, there are a lot of good resources for reverse proxy.  I’m simply writing this as a documentation trail for what I did, so 6 months from now I’ll remember.

NGINX installation and setup will not be covered here, only the changes I’ve made to get the system to send data to my back-end server hosting the NodeJS script for the Imgur Spark Bot.

First, all of my config is done within nginx.conf, so go edit that with your editor of choice (VIM FTW!).  In my case, I’ve chosen to serve it up with a URI of /spark, seems simple and logical.  I won’t go into the intricacies of reverse proxy, just know that some back-end applications need to understand if their URI is changing.  If so they generally have a way to tell the application that.  Why is this important?  Because any links or paths need to be adjusted accordingly.  Remember that the reverse-proxy is simply acting like a client to the back-end server on behalf of the person asking IT for content.  It has to know where to go and what links to use.

Can you strip the path off?  Sure (with rewrite) but honestly it’s much harder.  In our case I’m going to ensure NodeJS knows that there’s a URI path that matches NGINX, and we won’t have these issues.

Here is my relevant nginx.conf stanza.

location /spark {
  proxy_redirect off;
  proxy_set_header Host $host;
  proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

Easy enough?  I’m telling the publicly exposed NGINX server to send any request for /spark to the server at listening on port 90.  I’m adding some headers (in case I ever want to log where Spark requests come from, or more importantly to IP restrict them to known spark servers).

This is honestly all that’s required on the NGINX server, once it’s done we can proceed to finally creating some NodeJS code.

Employer Transitions

Inspired by Greg Ferro I have been considering, and subsequently made an employer transition recently.  This post will serve as some observations I’ve had after being in a new role for roughly 6 weeks.

  • It takes time to get comfortable with new surroundings
  • Things are different, but mostly similar everywhere in the Networking industry
  • Business and IT are frustrated at the state of our industry
  • ITIL is not fun

I’m also finding out that perspective is a funny thing.  I haven’t had many transitions in employer over the years, generally when I did it was a result of the business being purchased or closing, but it’s always been met with a measure of excitement and optimism.  Every time I’ve transitioned there are always great things, and not so great things.  This is the first time I’ve transitioned from larger to smaller, non-profit to for-profit, and yet the issues faced (political, outside of IT) are almost exactly the same.

If you haven’t read the article, go look for yourself and see if any of what Greg references sparks different perspectives.

Coding Fun – Imgur and Spark

One of the major themes recently in Network Engineering, and IT as a whole, is the pressure to learn coding as an engineer.  I do see this as a valuable skill, something to use to differentiate yourself, but on the whole enterprise IT isn’t going to see the value for at least 5 but more likely 10 years.

Ranting aside, I have been playing more recently with NodeJS to rewrite something I’ve written in python.  Why?  It gives me a chance to see how an algorithm (which is what true programming is all about) is able to be transitioned between languages.  In this series I’m going to detail how I accomplished this.

I’ve chosen to start to refresh my coding journey by having a little fun with Cisco Spark, specifically writing a Bot that will post animated gifs in response to direct questions.

Please note, I’m not going to touch on how Spark works, this is detailed very well in the link to their developer documentation.  This series will detail what I did to get everything working, the only pre-requisite is that you have already registered your Cisco Spark Bot, which is detailed in the developer documentation.

The first step in this is planning.  Like all good engineering projects, a little planning up front will save headaches down the road.

Below is a list of steps I plan to take in this series.

  • First, I will discuss webhooks and register one.
  • Second, I will setup an NGINX reverse-proxy server to handle the incoming webhook.
  • Third, I will detail getting NodeJS to listen for incoming requests, and handle them.
  • Fourth, I will detail using the Spark API to grab the detail of an incoming Message
  • Fifth, I will detail how to use the ImgUr API and NodeJS to search for and extract the path to an animated gif
  • Sixth, I will detail posting the new image using the Spark API, responding to either a direct message or the chat room/space it came from
  • Seventh and last, I will detail how to add a history function to your script, allowing the bot to take commands so it can delete old messages (useful in case something NSFW slips through)

Links will be updated off this master posting as I write the subsequent sections.


Coding Fun – Webhooks

What’s a web-hook?  Here’s the Wikipedia definition.

Clear as mud?

I try to imagine these new concepts in the terms a Network Engineer would understand.  Ever dealt with firewalls, and state?  Know what an out of state firewall request is?  That’s a web-hook.

Web-hooks allow servers to send data in reaction to an event(in our case, the event we care about is a  message).  Traditionally applications had to open up sockets and sit idle listening for data, it was the only mechanism for two-way communication between systems.  Modern systems with an API, allow you to register a web-hook, allowing you to subscribe to updates you want.

Web-hooks have positives, and one glaring negative.  The major positive is that you only get data you care about, and can predictably process it.  In our case we can subscribe only to messages, and be fairly confident that when we receive one it was designated for us and we need to do something with it.

The glaring negative, you have to be reachable all the time.  Since Spark runs over the Internet, this means I have to have something Internet exposed listening for this data and ready to respond.  Reverse proxy helps here, as I can expose something with a minimal Internet attack surface, and yet still have an internal system respond.

Enough talk, let’s get to how we use them.

First, review this link.  We will be using the Spark API to actually register our web-hook, so no coding is required yet.  As a bonus at the end of this article I’ll like to how this can be accomplished in either Python or NodeJS to register future web-hooks.

First thing to do, list web-hooks.  Click the relevant link within the API, and turn on test mode (Look for the science-y beaker and toggle it to On).  This test mode allows us to run the API queries directly from the browser, a really great feature for one-off requests.

Once you’ve logged into the spark portal, it is going to automatically fill out the authorization information (more on that later).  You can simply click Run and on the right you should see information returned(likely empty {} or an empty items array).

Yay, you just ran your first API query against spark.  I wish it was all this easy.  Now we need to register a web-hook.  Again we are concerning ourselves with how our script will receive messages and act on them, so naturally we want to register a web-hook related to messages.  Click on Create a Webhook (either from the overview link, or the menu on the left side of the API detail page).

Most things are going to auto-fill themselves, we concern ourselves with 4 properties under the Request Parameters section:


Oddly enough, these are the required parameters, funny how that works.

The name is purely for your sanity in listing them later, so you know what they do.  The resource is messages, and the event is created, which will likely be the defaults.

The targetUrl is the most important, this is what tells spark where to send data when it’s ready to send your bot a message.  Importantly, bots cannot just listen passively in a channel/space, they must be mentioned.  Private messages and mentions are what Spark is going to act on, and when it does it’ll send it to the Url you define here.

Once you’ve defined everything, click Run, again you should see the output in your browser(along with the request!).  This information is in JSON format.


If you get any other result than 200 / success, and a series of what look like random strings(they aren’t, it’s just unique identifiers) go back and ensure you have filled out everything required.

You should now be able to send your spark bot messages, without even a single line of code running!

Obviously we need to setup the actual listener (in our case NGINX will serve this) and the script in the back-end.  Those will be detailed in a future posting.