How to Send Data from C# to Splunk via the REST API

Splunk has a very extensive REST API – which is just a fancy way of saying that many of its capabilities are accessible via standard HTTP(S) requests. While much of the API is well documented, submitting data from C# to Splunk is kept a bit vague. Since I had to do that recently in order to implement an event generator for uberAgent I thought I might share my two solutions, both with and without the Splunk SDK for C#.

Single Event or Stream?

There are two ways to send events to Splunk: as single events or as a stream. The former is slightly more comfortable and better documented, but a lot (!) slower. If you have more than a few events per second to send, you need to use streaming or Splunk will not be able to keep up and events will simply not appear in the index (at least not in a timely fashion). You can check this by inspecting the max_age fields (expressed in seconds) in $SPLUNK_HOME\var\log\splunk\metrics.log.

Which Library: .NET framework, RestSharp, or the Splunk SDK for C#?

As I mentioned above a REST API uses standard HTTP(S) requests. As such it can be called from practically any programming language without additional libraries. Generally, I prefer not to be dependent on third-party code. However, accessing Splunk’s REST API from “naked” C# turned out to be so absurdly difficult (because of shortcomings of C#, mind you) that I quickly looked for an easier way. There is Splunk’s SDK for C#, of course, a set of routines designed to make the job (a lot) easier.

Splunk’s SDK only talks to Splunk, though. If you need a more generic solution or if you need to talk to other REST APIs, too, it makes sense to choose a generic REST library like RestSharp. Unfortunately, RestSharp does not support streaming (yet). For that reason, I present two routines in this article: one demonstrates submitting single events via RestSharp, the other implements streaming many events via Splunk’s SDK for C#.

Submitting a Single Event via RestSharp

using RestSharp;

/// <summary>
/// Send a single event to Splunk
/// </summary>
bool SubmitSingleEventToSplunk(string message, string host, string sourcetype)
{
   var client = new RestClient();
   client.BaseUrl = BuildUrl();
   client.Authenticator = new HttpBasicAuthenticator(_username, _password);

   string getParameters = string.Format("?source={0}&sourcetype={1}&host={2}&index={3}", HttpUtility.UrlEncode(_source), HttpUtility.UrlEncode(sourcetype),
                          HttpUtility.UrlEncode(host), HttpUtility.UrlEncode(_index));

   var request = new RestRequest("/services/receivers/simple" + getParameters, Method.POST);

   byte[] data = Encoding.UTF8.GetBytes(message);
   request.AddParameter("application/x-www-form-urlencoded", data, ParameterType.RequestBody);

   client.ExecuteAsync(request, response => { AsyncResponseHandler(response); });

   return true;
}


/// <summary>
/// Handle the post respose
/// </summary>
/// <param name="response"></param>
void AsyncResponseHandler (IRestResponse response)
{
   if (response.ErrorException != null)
   {
      Helpers.PrintError(string.Format("Posting failed with: {0}", response.ErrorException.Message));
   }
   if (response.ResponseStatus != ResponseStatus.Completed)
   {
      Helpers.PrintError(string.Format("Posting failed with: {0}", response.ErrorMessage));
   }
   if (response.StatusCode != HttpStatusCode.OK)
   {
      Helpers.PrintError(string.Format("Posting failed with: {0}\nresponse:\n{1}\nrequest:\n{2}", response.StatusDescription, response.Content, 
         Encoding.UTF8.GetString((byte[]) response.Request.Parameters[0].Value)));
   }
}


/// <summary>
/// Build an URL string from scheme, server and port
/// </summary>
string BuildUrl()
{
   return _scheme + "://" + _server + ":" + _port;
}

Streaming Multiple Events via Splunk’s SDK for C#

using Splunk;

/// <summary>
/// Submit multiple events to Splunk
/// </summary>
private bool SubmitMultipleEventsToSplunk (List<string> messages, string host, string sourcetype)
{
   try
   {
      // Login to Splunk
      var splunkService = new Service(_server, _port, _scheme);
      splunkService.Login (_username, _password);

      // Create a receiver to attach to (getting us the stream object)
      var splunkReceiver = new Receiver(splunkService);

      // Set host, source and sourcetype per stream
      var args = new Args();
      args.Add ("host", host);
      args.Add ("source", _source);
      args.Add ("sourcetype", sourcetype);
      using (var stream = splunkReceiver.Attach (_index, args))
      {
         foreach (var message in messages)
         {
            var bytes = Encoding.UTF8.GetBytes (message + "\r\n");

            stream.Write (bytes, 0, bytes.Length);
            stream.Flush();
         }
      }
      Console.Write ("\n");
   }
   catch (Exception e)
   {
      Helpers.PrintError("Error submitting to Splunk: " + e.Message);
   }

   return true;
}