Arduino and logging to the cloud

I participated in a lunch and learn today that demo’d the capabilities of logentries.com. It was impressive in the ease that you are able to parse, analyse and digest logging information. Once you have the log data being pushed to logentries.com there are any number of different ways that you can play with it. Seeing that, and knowing that we were going to push for it on my current project, I decided to take a look at it tonight from a slightly different angle. Instead of importing a nuget package and pumping data into it from that direction I figured I’d try to feed the service data at a much more raw level. Over the last few weeks of working with my Arduino I’ve come to appreciate how raw network communications can be…so why not just go there.

First thing I had to do was set up an account at logentries.com. It’s easy to do and it gives you 30 trial days of the full suite of features before reverting back to a free tier. There are a lot of different options for setting up logs once you’re in the system. At first I wanted to try sending log entries via a REST endpoint since it was what I knew best from my previous work with the Arduino. logentries offers a REST endpoint (HTTP PUT), but it’s being deprecated. So I looked at the other raw API options; Plain TCP/UDP and Token-based.

Plain TCP/UDP

This type of endpoint is all about sending TCP and/or UDP packets containing your log entries to an endpoint. The tricky thing with it is that it ties you to a specific IP address as the source of your log entries. The trick is that there is a 15 minute window which links the incoming message and it’s source IP address to your account’s log. Not a horrible thing, but a setup restriction none-the-less. More information on how it works can be found here.

Token-based

Like the Plain TCP/UDP option, all log traffic is sent to an endpoint via TCP and/or UDP. The difference is that your log data will contain a token and that will be used to tie the messages you send to the account you’re using. This is much easier than worrying about getting the right IP address linked to the account in the first 15 minutes of the log’s life. I chose this option because of the simplicity. More info on it here.

Arduino code

I’ve been playing with sending data from my Arduino to Azure Mobile Services (more on that another time) over the last few weeks. As a result I had some pretty good code written (mostly borrowed from Adafruit examples) to hook up my Arduino to WiFi. I bought a CC3000 shield from Adafruit for my Uno and made use of it for this project too. There’s a lot of boilerplate code to get the WiFi up and running, but that’s not the interesting part. What you’re here to see is how the data is sent to https://logentries.com/logentries.com after a connection has been established.

void Log(char logEntry[100]){
  t = millis();
  if (!logger.connected()){
    Serial.println("Connecting logger");
    do {
      logger = cc3000.connectTCP(log_ip, 80);
    } while ((!logger.connected()) && ((millis() - t) < connectTimeout));
  }
  Serial.println("logger connected");

  if (logger.connected()) {
    Serial.println("logging");
    logger.fastrprint(LOG_TOKEN);
    Serial.print(LOG_TOKEN);
    logger.fastrprint(" ");
    logger.fastrprintln(logEntry);
    Serial.println(logEntry);
    logEntry[0] = 0;
  }
}

There are a couple of things going on in this method.

1. On line 6 we establish a TCP connection to the logentries.com endpoint (data.logentries.com).
2. If the connection to the endpoint is made successfully we move on to sending the log entry. Lines 13, 15 and 17 make this happen. “logger” represents the connection to the endpoint and we call .fastrprint(….) to send data and .fastrprintln(….) to send data with a line ending.

All we need to send out in our stream to the endpoint is the log data we want to include and the token we got when we created the log on the logentries.com website. By sending the token as a .fastrprint, the blank as a .fastrprint, and the message as a .fastrprintln we’re essentially sending all three of these pieces of information as one line to the endpoint.

The output

Here’s what things look like when you look on the website.

Now you can start making use of logentries’ tagging and reporting functionality to understand how your Arduino code is working.

Source code for the sample project is available on github: https://github.com/dbelcham/logentries_arduino