The Open Source Observability Landscape

Storing and Accessing Telemetry

Are you feeling overwhelmed by the huge explosion of choices for observability and monitoring in open source? How you are expected to develop and deploy applications is changing rapidly — from monoliths to microservices and more, complexity keeps growing. Luckily, however, there’s been a renaissance in open source innovations as projects including ELK, Prometheus, and Jaeger grow in popularity and usage. You might have even heard of service meshes like Istio that can generate telemetry automatically. How do you make sense of all these tools and how do they stack up when it comes to solving problems quickly? This guide is written to help you make sense of the most popular open source methods of storing and accessing telemetry, and how they relate to Honeycomb. In this paper, we’ll cover the storage systems for persisting and accessing observability data — there are also open source projects such as OpenTelemetry that define a format for exporting data to these storage systems, but we’ll save discussing those for another day.

Intro

It used to be a simpler time — you had fewer services, often deployed to only one powerful server, that you were responsible for. Most of the things that could go wrong with these monoliths were well understood. You logged slow database queries as they happened and added MySQL indexes. You spotted network issues easily with high level metrics. You monitored disk space and memory usage remedied shortages when needed. When something went wrong, the path to finding out what happened was straightforward — likely you would access a shell on a server, read the logs, deduce what was happening and act accordingly to fix it.

Eventually, however, the Internet became a victim of its own success — traffic had to be served to more people, teams had to scale out their development, and 1 novel challenges began to brew. Given Linux’s critical role in enabling large scale web applications in the first place, it’s no surprise that the open source heritage of the infrastructure carried forward to new troubleshooting tools. Programmers solving their own problems began to build out the next generation of tooling to solve these complicated new issues and you stand to benefit by leveraging their work and experience.