How to Send Data from C# to Splunk via the REST API

Splunk has a very extensive REST API – which is just a fancy way of saying that many of its capabilities are accessible via standard HTTP(S) requests. While much of the API is well documented, submitting data from C# to Splunk is kept a bit vague. Since I had to do that recently in order to implement an event generator for uberAgent I thought I might share my two solutions, both with and without the Splunk SDK for C#.

Single Event or Stream?

There are two ways to send events to Splunk: as single events or as a stream. The former is slightly more comfortable and better documented, but a lot (!) slower. If you have more than a few events per second to send, you need to use streaming or Splunk will not be able to keep up and events will simply not appear in the index (at least not in a timely fashion). You can check this by inspecting the max_age fields (expressed in seconds) in $SPLUNK_HOME\var\log\splunk\metrics.log.

REST for the wicked

Which Library: .NET framework, RestSharp, or the Splunk SDK for C#?

As I mentioned above a REST API uses standard HTTP(S) requests. As such it can be called from practically any programming language without additional libraries. Generally I prefer not to be dependent on third-party code. However, accessing Splunk’s REST API from “naked” C# turned out to be so absurdly difficult (because of shortcomings of C#, mind you) that I quickly looked for an easier way. There is Splunk’s SDK for C#, of course, a set of routines designed to make the job (a lot) easier.

Splunk’s SDK only talks to Splunk, though. If you need a more generic solution or if you need to talk to other REST APIs, too, it makes sense to choose a generic REST library like RestSharp. Unfortunately RestSharp does not support streaming (yet). For that reason I present two routines in this article: one demonstrates submitting single events via RestSharp, the other implements streaming many events via Splunk’s SDK for C#.

Submitting a Single Event via RestSharp

using RestSharp;
 
/// <summary>
/// Send a single event to Splunk
/// </summary>
bool SubmitSingleEventToSplunk(string message, string host, string sourcetype)
{
   var client = new RestClient();
   client.BaseUrl = BuildUrl();
   client.Authenticator = new HttpBasicAuthenticator(_username, _password);
 
   string getParameters = string.Format("?source={0}&sourcetype={1}&host={2}&index={3}", HttpUtility.UrlEncode(_source), HttpUtility.UrlEncode(sourcetype),
                          HttpUtility.UrlEncode(host), HttpUtility.UrlEncode(_index));
 
   var request = new RestRequest("/services/receivers/simple" + getParameters, Method.POST);
 
   byte[] data = Encoding.UTF8.GetBytes(message);
   request.AddParameter("application/x-www-form-urlencoded", data, ParameterType.RequestBody);
 
   client.ExecuteAsync(request, response => { AsyncResponseHandler(response); });
 
   return true;
}
 
 
/// <summary>
/// Handle the post respose
/// </summary>
/// <param name="response"></param>
void AsyncResponseHandler (IRestResponse response)
{
   if (response.ErrorException != null)
   {
      Helpers.PrintError(string.Format("Posting failed with: {0}", response.ErrorException.Message));
   }
   if (response.ResponseStatus != ResponseStatus.Completed)
   {
      Helpers.PrintError(string.Format("Posting failed with: {0}", response.ErrorMessage));
   }
   if (response.StatusCode != HttpStatusCode.OK)
   {
      Helpers.PrintError(string.Format("Posting failed with: {0}\nresponse:\n{1}\nrequest:\n{2}", response.StatusDescription, response.Content, 
         Encoding.UTF8.GetString((byte[]) response.Request.Parameters[0].Value)));
   }
}
 
 
/// <summary>
/// Build an URL string from scheme, server and port
/// </summary>
string BuildUrl()
{
   return _scheme + "://" + _server + ":" + _port;
}

Streaming Multiple Events via Splunk’s SDK for C#

using Splunk;
 
/// <summary>
/// Submit multiple events to Splunk
/// </summary>
private bool SubmitMultipleEventsToSplunk (List<string> messages, string host, string sourcetype)
{
   try
   {
      // Login to Splunk
      var splunkService = new Service(_server, _port, _scheme);
      splunkService.Login (_username, _password);
 
      // Create a receiver to attach to (getting us the stream object)
      var splunkReceiver = new Receiver(splunkService);
 
      // Set host, source and sourcetype per stream
      var args = new Args();
      args.Add ("host", host);
      args.Add ("source", _source);
      args.Add ("sourcetype", sourcetype);
      using (var stream = splunkReceiver.Attach (_index, args))
      {
         foreach (var message in messages)
         {
            var bytes = Encoding.UTF8.GetBytes (message + "\r\n");
 
            stream.Write (bytes, 0, bytes.Length);
            stream.Flush();
         }
      }
      Console.Write ("\n");
   }
   catch (Exception e)
   {
      Helpers.PrintError("Error submitting to Splunk: " + e.Message);
   }
 
   return true;
}

, ,

4 Responses to How to Send Data from C# to Splunk via the REST API

  1. Gavin Stevens March 20, 2014 at 05:06 #

    List messages inferring a simple list of strings? What if I want to Model a complex object and Store instances of it in Splunk? This Stream of Messages seems a bit crude..

    • Helge Klein March 20, 2014 at 11:05 #

      Splunk stores events as simple text, so your complex objects need to be converted to strings anyway. The article focuses on how to send data to Splunk. It is assumed that you know enough C# to be able to convert your objects to string yourself.

      Also, there is more than one possible way to represent objects as text (json, xml, key-value pairs, CSV). You may want to decide which to use yourself – outside of the routine that merely sends the data off.

  2. Glenn Block August 6, 2014 at 09:11 #

    Hi Helge

    We just launched a new logging library that is designed to make it really easy to send events to Splunk from .NET applications in an efficient matter over UDP. It includes support for .NET tracing and for doing semantic logging with SLAB.

    Docs are here. http://dev.splunk.com/view/splunk-loglib-dotnet/SP-CAAAEX4

    We also have a new VS Plugin which helps you to wire this up. http://dev.splunk.com/view/splunk-extension-vs/SP-CAAAEXZ

    Would love for you to try it out.

  3. Connor September 19, 2014 at 18:26 #

    Awesome post Helge.

    Would you mind following up with an example of using TCP to send data to Splunk?

    I’m currently streaming multiple events with the receiver.attach method, but it’s severely limited by the throughput averaging at about 10k events per minute.

Leave a Reply