Coding fun – API?

Fair warning, this is probably going to be a very long post. TLDR; APIs are useful.

Web-Hook is setup, NGINX is proxying and forwarding, NodeJS is listening, now we need to actually do something with the Spark data that is being received.

Remember the API?  Sure you do (here it is).  We will be using this extensively so pull it up in another window so you can reference it.

Let’s mock-up what we’re trying to accomplish.

  • Spark notifies us of a message
  • We grab the details of the message
  • We process those details

Easy right?  Wait, we have to grab details, why doesn’t spark just send us the details?  I’m not exactly sure, but likely so nothing super sensitive is being sent to a site that can potentially be hijacked.  So what are they sending us? It looks a little like this:

{ 
 id: 'garbldegoop',
 name: 'ImgurBot WebHook',
 targetUrl: 'https://example.com/spark',
 resource: 'messages',
 event: 'created',
 orgId: 'garbledegoop',
 createdBy: 'garbldegoop',
 appId: 'garbldegoop',
 ownedBy: 'creator',
 status: 'active',
 created: '2017-09-07T13:23:23.181Z',
 actorId: 'garbldegoop',
 data: 
 { 
  id: 'important garbldegoop',
  roomId: 'more important garbldegoop',
  roomType: 'direct',
  personId: 'garbldegoop',
  personEmail: 'email@email.com',
  created: '2017-09-08T15:24:52.948Z' 
 } 
}

Fill in a bunch of random strings where you see garbldegoop, each one is unique, and each is unique per message.  I’ve obfuscated mine so that some important details can be masked, but regardless we will be using code to break this big thing apart, and grab some details.

What fields do we need?  Depends what we’re going to do.  First we need to infer a few things from what we just received.

  • Where did this message come from?
    • Was it a Room/Space?
      • If it was, how can I tell where that is?
    • Was it an individual via a Direct Message?
      • What is their email address, or ID?
  • Where is the message itself?

Most APIs break all this stuff up so you can only grab the things you want, Spark is no different.  Using my example above, I know this message came in via a room.  How?

roomType: 'group'

Easy?  What’s it look like if it’s simply a message from a user?

roomType: 'direct'

Anyone can do this stuff, with that one field we can determine how our reply needs to happen.  But how do we get it?  Fun fact, Spark sent it to us directly.  Our code already handles it.  Remember app.all from the previous post?  No?  Here it is:

app.all(..) {
}

That doesn’t tell us much, let’s expand it and review what’s going on.

app.all('/spark', (req, res) => {
 res.send();
 console.log(req.body);
});

Simple.  Add this code to your app.all stanza and run it.  You can direct message your bot and it should dump the contents of the Spark web-hook to the console.  Welcome to debugging, printing things until you get what you want.  Buy what’s going on here?  Let’s detail it:

app.all('/spark', (req, res) => { 
  .. 
});

So, API calls use GET, or POST, or PUT, or DELETE, or UPDATE right?  That Express module we installed yesterday, it can handle them individually, or all at once.  Why would you do that?  It was designed to actually create APIs, we’re not doing that directly, instead we’re borrowing that functionality to get NodeJS listening to something.  By using the .all we are telling Node and Express to ignore what type of call comes in (through the reverse-proxy, remember that thing?)

When we registered the web-hook, we gave it a URI path (hint, /spark) so it’s happily passing it through.  We have to tell Express to listen for a URI (app.all(‘/spark’…)).  Sure I could have used ‘*’ here, but that would have opened this thing up to ANY web request making it through the proxy, I like security too, so lets just make sure things line up as much as possible.

Now, if you read the documentation for express, it’s actually expecting the next part to be a callback.  What is a callback?

Wouldn’t it be fun if we could just tell computers how to handle data on the fly?  We didn’t used to, programmers generally had to hard-code every possible way data could be handled, no flexibility in that.

A callback is a function that YOU write, telling some other function how to handle data.  Welcome to recursion(another fun, but very abstract CompSci topic).  This solves our future data problem.  All a programmer has to know is what data is being returned, and can act on it.

app.all('/spark', (req, res) => {
 ..
});

Everything in bold is a function, the callback.  Here’s how we could write it differently

app.all('/spark', callbackFunction);

For readability it might make sense to do this, but the more you get used to NodeJS the more you’re going to realize almost everything uses a callback.  It would get really tedious to write these out and substitute them everytime you wanted to implement a callback, right?  We are coders, we like to do things lazy and easy, so write the function in-place.

Speaking of, see the funny => thing in our callback?  That’s a really interesting way to create functions in NodeJS.  Remember, we like to be lazy.  The link I gave you is a very thorough example of why we use these, read it and research “arrow” functions.  This will help you understand some of why we use them (other than laziness!).

So, Spark is sending us some data, our app.all(..) is receiving that data, and we have some kind of callback function thing doing stuff.  What’s it doing?

 res.send();

Back to relating things to Network Engineers.  Ever typed a command wrong on a cisco IOS cli prompt?  What’s the first thing you always forget to disable, which locks the session up until DNS times out?  no ip domain-lookup right?

Spark is sending us information.  It’s using HTTP to send us that data.  HTTP expects to be answered, even if it’s a return of blank data.  We don’t want to be the bad dns-lookup.  Spark needs to be answered, so we do it.  We send back a reply, it’s blank, the equivalent of sending an HTTP status of 200 with nothing in it.  Spark is happy, we’re happy.

console.log(req.body);

Take whatever Spark sent us (req.body) and log it to the console.  I wish it were this simple, remember earlier we downloaded a new module (body-parser) and included it?

const bodyparser = require('body-parser');
..
app.use(bodyparser.json());
app.use(bodyparser.urlencoded({ extended: true }));

Spark is using a POST method to deliver our information.  POST embeds all it’s data in the body of an HTML document.  Sometimes this is plaintext, urlencoded, obfuscated, whatever.  Body-parser includes the magic needed to interpret all this stuff and get it into a readable format we can use, in our case Spark is sending us JSON so we’re processing it.

Go read the documentation for body-parser for more information, it’s really good at explaining this.

So req.body contains that formatted data, we dump it to screen.

I told you this would be long, go get a drink, stretch your legs, and come back.  I’ll wait.

 

—–

 

Back?  Good, let’s start working with the spark API itself, it gave us information, it’s only fair we USE it for something.

We’re working with JSON.  Probably pretty good we’re using JavaScript to process it, being that it was created for that language originally.  JSON is really easy to process within NodeJS, you assign a variable some java script and magic happens in the background.

We’re going to rewrite app.all now:

app.all('/spark', (req, res) => {
  res.send();
  var sparkDetail = req.body;
  var messageId = sparkDetail.id;
  spark.getMessage(messageID, (messageContents) => {
   console.log(messageContents);
  });
);

As a refresher, here’s the JSON detail that spark sends us

{ 
 id: 'garbldegoop',
 name: 'ImgurBot WebHook',
 targetUrl: 'https://example.com/spark',
 resource: 'messages',
 ...
}

Breaking down our changes to app.all

var sparkDetail = req.body;

Creating a new variable, named sparkDetail.  It contains the entire JSON that spark sends us.

var messageId = sparkDetail.id;

I told you earlier, when you assign JSON to a variable in NodeJS it just magically knows how to handle it.  We did that with the body, Spark only sends JSON in the body.  JSON is simply a series of variables and values being sent to us.  We need to extract data from those variables.  Specifically we need the variable called ID, and the contents of it.  It’s the very first thing Spark sends us in the JSON document.  Since sparkDetail contains the entire JSON data, I can simply request the ID variable by appending .id to it.

If I wanted to get the name, it’d be sparkDetail.name, the targetUrl would be sparkDetail.targetUrl, and so on…

spark.getMessage(messageId, <<CALLBACK>>);

I ommited the callback for clarity, we are calling a new function here (that doesn’t exist yet!) in the module spark(we haven’t created it), the function is getMessage.  It expects the ID of the message we want to get, and will return some data.  That data will be handled by a callback.

I’ll rewrite it with the callback

spark.getMessage(messageID, (messageContents) => {
 console.log(messageContents);
});

Go lookup arrow functions if you didn’t earlier, understand them before trying to parse this out.  The callback function written out explicitly is:

function (messageContents) {
  console.log(messageContents);
}

I know, I know, arrow functions aren’t full functions.  Go ahead and comment and tell me that, otherwise get over it and move on if you understand that concept.  The getMessage function will return data for us that gets put into the variable called messageContents.  We then log it to console.

Did you catch what’s wrong?  The spark module isn’t loaded, in fact we haven’t created it yet.  Let’s do that now.  Edit a new file and name it spark.js, here’s the barebones contents:

const request = require('request')

var getMessage = (msgId, callback) => {
  // get spark msgId
};

var postMessage = (body, callback) => {
  // post spark message
};

var deleteMessage=(msgId)=>{
  //
};

module.exports = {
  getMessage,
  postMessage,
  deleteMessage
};

Really only one new thing to learn in this, what does module.exports do?  Well, when you create a modular program you need to let your main program know what functions or variables you are giving it access to.  In our case we’re writing all the logic to interact with the Spark API directly in this module, so we need to expose the functions we want the consumer to use.  module.exports does that for us.

Back to the Spark API, this time we are looking at messages.  Specifically we are going to use the GET method specifying the messageId, let’s rewrite our getMessage function and then go through some of the highlights

var getMessage = (msgId, callback) => {
  var sparkAPI = 'https://api.ciscospark.com/v1/messages/';

  var options = {
    url: sparkAPI + msgId,
    headers: {
      'Authorization': 'Bearer <YOUR BOT KEY>';
    },
    json: true
  };

  request.get(options, (error, response, data) => {
    if(!error && response.statusCode == 200) {
      callback(data);
    }
  };
};

First we define a couple of variables, the first is obvious, the path to the Spark API method we want to call.  The second however needs a little discussion.

RESTful APIs generally require some sort of authentication.  Some use OATH, some don’t.  All of them generally expect some kind of token to be presented, to prove you are who you are, or to track your usage (billing!).  Spark is no exception, the only difference is that it generates your bearer token when you create your bot.  You did write that down right?  No, well good thing you can use the Test mode of the API to get it.  When you put it into Test mode you should see the Bearer <KEY> line show up, just copy it and substitute it where appropriate.

Let’s run this code and see what our output is.  Oh, we need to update our server code to call it and register the callback.

const express = require('express');
const bodyparser = require('body-parser');

const spark = require('./spark.js');

var app = express();
app.use(bodyparser.json());
app.use(bodyparser.urlencoded({ extended: true }));

app.all('/spark', (req, res) => {
 res.send();
 //console.log("**", req.body);
 spark.getMessage(req.body.data.id, (messageDetail) => {
    console.log(messageDetail);
 });
});

app.listen(90);

Add this to your server file, and run it.  You should be able to Direct Message your bot’s name, and the output on the screen will be the details of that message.

That will end this one, next we’ll work on using the Message data to search Imgur.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s