Skip to content

nitinka/D-S-Trace

Repository files navigation

D-S-TRACE ( D-Stress )

Distributed tracing in complex distributed application environment is a very basic and most needed capability. I have used available solutions like Zipkin, i must admit not a quickest solution(niether to deploy nor to use) to get started with.
With D-S-Trace I intent to provide distributed tracing platform which you can setup in little time and get started.
To learn more about philosophy of distributed tracing : http://www.slideshare.net/dkuebrich/distributed-tracing-in-5-minutes

Alt Image

DSTrace consist of 3 components :

1) dstrace-publisher : applications should include this library to start sending various events that they want to trace. These events are published to Redis or directly to dstrace-archive depending upon configuration. Publishing via Redis makes the deployment more reliable.
2) dstrace-archive : receive published events (directly from publisher as http call or subscribed message from Redis topic) and archive message to elastic search.
3) dstrace-console : Set of rest end points and UI to search, relate and visualize distributed traces.

Deployment:

1) Install Redis: Follow http://redis.io/topics/quickstart
2) Install elasticsearch:
1) Download elastic Search : https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-0.90.7.zip
2) unzip
3) cd bin
4) ./elasticsearch -f
5) curl -XPOST 'http://localhost:9200/trace' –d '
{
  "mappings": {
    "event": {
      "properties": {
        "application": {
          "type": "string",
          "index": "not_analyzed"
        },
        "businessUnit": {
          "type": "string",
          "index": "not_analyzed"
        },
        "eventId": {
          "type": "string",
          "index": "not_analyzed"
        },
        "traceId": {
          "type": "string",
          "index": "not_analyzed"
        },
        "eventName": {
          "type": "string",
          "index": "not_analyzed"
        },
        "host": {
          "type": "string",
          "index": "not_analyzed"
        },
        "spanName": {
          "type": "string",
          "index": "not_analyzed"
        },
        "spanId": {
          "type": "string",
          "index": "not_analyzed"
        },
        "tags": {
          "type": "object"
        },
        "timestamp": {
          "type": "long"
        }
      }
    },
    "businessUnit": {
      "properties": {
        "name": {
          "type": "string",
          "index": "not_analyzed"
        },
        "applications": {
          "type": "nested",
          "index": "not_analyzed",
          "properties": {
            "name": {
              "type": "string"
            },
            "operations": {
              "type": "nested",
              "index": "not_analyzed",
              "properties": {
                "name": {
                  "type": "string",
                  "index": "not_analyzed"
                }
              }
            }
          }
        }
      }
    }
  }
}'
3) Install D-S-Trace Services :
1) git clone https://github.com/nitinka/D-S-Trace.git 
2) cd D-S-Trace
3) mvn clean compile package
4) cd dstrace-archive
5) mvn exec:java
6) Open another console
7) cd dstrace-console
8) mvn exec:java
4) Done

Getting Started With Tracing:

1) Include Publisher Library in your application : allows users to publish events to dstrace-archive service.
<repositories>
    <repository>
        <id>nitinka.mvn.repo</id>
        <url>https://github.com/nitinka/mvn-repo/raw/master</url>
        <!-- use snapshot version -->
        <snapshots>
            <enabled>true</enabled>
            <updatePolicy>always</updatePolicy>
        </snapshots>
    </repository>
</repositories>

<dependency>
    <groupId>nitinka.dstrace.publisher</groupId>
    <artifactId>dstrace-publisher</artifactId>
    <version>0.0.4</version>
</dependency> 

2) Initialize Publisher/Trace Configuration :Initialize Tracer during the initialization of your application.

TracerConfiguration configuration = new TracerConfiguration();
        configuration.
                setEnabled(true).
                setBusinessUnit("bu1").
                setApplication("app1").
                setEventQueueSize(100000).
                setHost("localhost").
                setSamplePercentage(100).
                setPublishConfiguration(new EventPublishConfiguration().
                        setEventPublishDelay(5000).
                        setEventPublishSize(100).
                        setEnqueueFailedMessages(false).
                        setPublisherClass("nitinka.dstrace.publish.RedisPublisherImpl").
                        setPublisherConfig(new HashMap<String, Object>(){
                            {
                                put("redisHost","localhost"); 
                                put("redisPort","trace-events");
                            }
                        }));

        Tracer.initialize(configuration);

3) Publish events from your app :Use Trace class to publish events

        Tracer.setCurrentTraceId(UUID.randomUUID().toString()); // Should be unique transaction id shared across application
        Tracer.startSpan("span1",null, null);
        Thread.sleep(100);
        Tracer.startSpan("span1.1", null, null);
        Thread.sleep(100);
        Tracer.startSpan("span1.1.1", null, null);
        Thread.sleep(100);
        Tracer.startSpan("span1.1.1.1", null, null);
        Thread.sleep(100);
        Tracer.startSpan("span1.1.1.1.1", null, null);
        Thread.sleep(1000);
        Tracer.endSpan();
        Thread.sleep(1000);
        Tracer.endSpan();
        Thread.sleep(1000);
        Tracer.endSpan();
        Thread.sleep(1000);
        Tracer.endSpan();
        Thread.sleep(1000);
        Tracer.endSpan();

4) Get the basic trace view :Use trace ID to get the complete distributed transaction

http://localhost:8888/index.htm?traceId=5993126e-24e3-4352-99f4-9a425b6fadd9

Alt Image 5) Get the raw trace(json) :Use trace ID to get the complete distributed transaction

http://localhost:8888/dstrace-console/traces/5993126e-24e3-4352-99f4-9a425b6fadd9
{
  "traceId": "5993126e-24e3-4352-99f4-9a425b6fadd9",
  "startTime": 1384939975603,
  "endTime": 1384939981010,
  "durationMS": 5407,
  "firstSpan": "1",
  "lastSpan": "1",
  "spans": {
    "1": {
      "spanId": "1",
      "spanName": "span1",
      "events": [
        {
          "businessUnit": "bu1",
          "application": "app1",
          "host": "localhost",
          "spanName": "span1",
          "traceId": "5993126e-24e3-4352-99f4-9a425b6fadd9",
          "eventId": "a45cae4f-4f86-40ab-82d7-f7f584bfaf05",
          "spanId": "1",
          "parentSpanId": null,
          "eventName": "Start",
          "timestamp": 1384939975603,
          "tags": {
            
          }
        },
        {
          "businessUnit": "bu1",
          "application": "app1",
          "host": "localhost",
          "spanName": "span1",
          "traceId": "5993126e-24e3-4352-99f4-9a425b6fadd9",
          "eventId": "b668d9b5-7182-42c6-89ea-b4fc0c99dfc0",
          "spanId": "1",
          "parentSpanId": null,
          "eventName": "End",
          "timestamp": 1384939981010,
          "tags": {
            
          }
        }
      ],
      "childSpans": {
        "2": {
          "spanId": "2",
          "spanName": "span1.1",
          "events": [
            {
              "businessUnit": "bu1",
              "application": "app1",
              "host": "localhost",
              "spanName": "span1.1",
              "traceId": "5993126e-24e3-4352-99f4-9a425b6fadd9",
              "eventId": "328289a9-8660-4bbc-aaee-03ac3d6e71b3",
              "spanId": "2",
              "parentSpanId": "1",
              "eventName": "Start",
              "timestamp": 1384939975704,
              "tags": {
                
              }
            },
            {
              "businessUnit": "bu1",
              "application": "app1",
              "host": "localhost",
              "spanName": "span1.1",
              "traceId": "5993126e-24e3-4352-99f4-9a425b6fadd9",
              "eventId": "3745dcd9-2077-4373-bd6e-245eacf28b8e",
              "spanId": "2",
              "parentSpanId": "1",
              "eventName": "End",
              "timestamp": 1384939980009,
              "tags": {
                
              }
            }
          ],
          "childSpans": {
            "3": {
              "spanId": "3",
              "spanName": "span1.1.1",
              "events": [
                {
                  "businessUnit": "bu1",
                  "application": "app1",
                  "host": "localhost",
                  "spanName": "span1.1.1",
                  "traceId": "5993126e-24e3-4352-99f4-9a425b6fadd9",
                  "eventId": "ee9391f1-add7-48e6-a287-33152723ea11",
                  "spanId": "3",
                  "parentSpanId": "2",
                  "eventName": "Start",
                  "timestamp": 1384939975805,
                  "tags": {
                    
                  }
                },
                {
                  "businessUnit": "bu1",
                  "application": "app1",
                  "host": "localhost",
                  "spanName": "span1.1.1",
                  "traceId": "5993126e-24e3-4352-99f4-9a425b6fadd9",
                  "eventId": "fb9ca561-41c7-4338-959d-92ce28877860",
                  "spanId": "3",
                  "parentSpanId": "2",
                  "eventName": "End",
                  "timestamp": 1384939979009,
                  "tags": {
                    
                  }
                }
              ],
              "childSpans": {
                "4": {
                  "spanId": "4",
                  "spanName": "span1.1.1.1",
                  "events": [
                    {
                      "businessUnit": "bu1",
                      "application": "app1",
                      "host": "localhost",
                      "spanName": "span1.1.1.1",
                      "traceId": "5993126e-24e3-4352-99f4-9a425b6fadd9",
                      "eventId": "355bf2a1-2fb1-490c-8e08-8085ba11f86a",
                      "spanId": "4",
                      "parentSpanId": "3",
                      "eventName": "Start",
                      "timestamp": 1384939975905,
                      "tags": {
                        
                      }
                    },
                    {
                      "businessUnit": "bu1",
                      "application": "app1",
                      "host": "localhost",
                      "spanName": "span1.1.1.1",
                      "traceId": "5993126e-24e3-4352-99f4-9a425b6fadd9",
                      "eventId": "076071cb-7592-4176-b430-cf36aabe8399",
                      "spanId": "4",
                      "parentSpanId": "3",
                      "eventName": "End",
                      "timestamp": 1384939978008,
                      "tags": {
                        
                      }
                    }
                  ],
                  "childSpans": {
                    "5": {
                      "spanId": "5",
                      "spanName": "span1.1.1.1.1",
                      "events": [
                        {
                          "businessUnit": "bu1",
                          "application": "app1",
                          "host": "localhost",
                          "spanName": "span1.1.1.1.1",
                          "traceId": "5993126e-24e3-4352-99f4-9a425b6fadd9",
                          "eventId": "6edffcca-f351-4b5e-b68d-9e33678c04ce",
                          "spanId": "5",
                          "parentSpanId": "4",
                          "eventName": "Start",
                          "timestamp": 1384939976006,
                          "tags": {
                            
                          }
                        },
                        {
                          "businessUnit": "bu1",
                          "application": "app1",
                          "host": "localhost",
                          "spanName": "span1.1.1.1.1",
                          "traceId": "5993126e-24e3-4352-99f4-9a425b6fadd9",
                          "eventId": "9724f3d2-d314-49e0-8874-95891144d1f3",
                          "spanId": "5",
                          "parentSpanId": "4",
                          "eventName": "End",
                          "timestamp": 1384939977007,
                          "tags": {
                            
                          }
                        }
                      ],
                      "childSpans": {
                        
                      },
                      "duration": 1001
                    }
                  },
                  "duration": 2103
                }
              },
              "duration": 3204
            }
          },
          "duration": 4305
        }
      },
      "duration": 5407
    }
  }
}

About

Simplest Distributed Tracing Platform.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages