Observability

Analyse Loki logs with an agent that can read your code.

Finding the log line is half the job. The other half is working out what emitted it and why.

Download for macOS
v0.1.33 · .dmg · Apple Silicon & Intel
Signed & notarized by Apple · opens without a Gatekeeper warning
sha256 698955a0187bc039f4c74f5d05a9f10fbb27376a45788a0a241d1326b73873c7
Download for Windows instead
$curl -fsSL https://lucidtrain.com/install.sh | sh

LogQL, generated

LogQL's stream selectors and parser expressions are more awkward to write from memory than PromQL, particularly when you need to extract a field from a JSON line and filter on it. Describing what you are looking for and getting the query is a real saving during an incident, which is precisely when nobody wants to read documentation.

The jump to code

A log line is a string emitted from a specific place. With the repository open the agent can find that place, which turns an unfamiliar error message into a location and a surrounding context. That step is normally a manual grep against a message that has been reformatted since it was written.

Patterns across many lines

The useful question is often not what one line says but whether a class of them correlates with something: a deploy, one tenant, one endpoint. That is analysis over the result set rather than a lookup, and it is the kind of thing that is tedious by hand and quick to ask for.

Local matters here most

Of everything in this category, logs are the data most likely to contain something sensitive, frequently by accident. Running against a local model means the lines stay on your machine while you analyse them.

Ask for it

shell
# Find the errors in the checkout service in the last hour, group them by
# message, and show me the code that emits the most common one.
FAQ

Questions

What does it need to connect?

A Loki endpoint and credentials, stored in the OS keychain. Grafana's Loki data source works too.

Can it correlate logs with metrics?

Yes, when both are connected. Correlating an error spike with a latency change is one of the more useful things to ask, and one of the more tedious to do by hand.

Does it send my logs anywhere?

With a local model, no. Given what logs tend to contain, that is the configuration worth using here.

Related