Distributed log parsers face key challenges but can be benchmarked with new metrics
The claim
The authors assert that distributed parsing can be effectively evaluated using metrics and aggregation techniques across many configurations
Published as: “Too many clients spoil the logs: distributed parsing, aggregation techniques and metrics”
So what
The results guide practitioners on when aggregation is worthwhile (e.g., many clients, diverse templates) and warn of added communication, privacy, and complexity costs.
The five-minute version
The paper finds that increasing the number of clients or decreasing per-client data volume degrades parser performance, especially for LogBatcher, while aggregation techniques can improve overall performance but the gains vary by parser and metric. The evidence comes from 6,720 distributed parsing experiments (40,320 results) on public HDFS and BGL datasets, but the abstract is missing so details are drawn from the introduction and results sections.
What they found 3
- Aggregation techniques can improve overall performance in distributed settings.
“RA2: Aggregation techniques can improve overall performance in distributed settings.”
- LogBatcher benefits more from concatenation-based aggregation than other parsers.
“LogBatcher, which is more sensitive to data distribution, benefits more from the concatenation-based aggregation,”
- Concatenation is the least computationally complex aggregation method and is recommended for future work.
“we consider it a more suitable candidate to be use in future work.”
What undercuts it the paper’s own results
- Other parsers, such as Drain and Brain, yield only limited gains or none at all, particularly on datasets like HDFS.
- techniques such as concatenation increase the number of templates but do not necessarily lead to information gain.
Stated limits 3
- The ground truth templates are used to evaluate how well the different parsers perform under various configurations. However, these ground truth templates can include minor human mistakes that introduce noise into the final results.
- In this study, we aim to report the results obtained from a large and diverse set of parsers, since relying on a single type of parser could introduce bias into our overall findings. Nevertheless, parser algorithm development is an active and evolving field, and it is possible that the conclusions drawn in this paper may no longer hold true when future parsers are considered.
- For our evaluation we utilize the LLM-parser LogBatcher (Xiao et al., 2024). Following previous research (Beck et al., 2025; Jiang and et al., 2024b) we note that the pretraining phase of the LLMs could have exposed them to the log data set used in this study, giving them an advantage for log parsing.
What’s new here 4
- Systematic benchmarking of multiple log parsers under varying numbers of clients and data volumes in a distributed setting.
- Introduction of a taxonomy of parsing metrics tailored for federated environments.
- Evaluation of several lightweight aggregation methods (concatenation, merging) for distributed parsers, a topic previously unexplored.
- Large-scale empirical study comprising 6,720 experiments and 40,320 individual results.
Artifacts & methods 10
- LogBatchersystem
- Drainsystem
- Brainsystem
- AELsystem
- LenMAsystem
- concatenation aggregationprimitive
- set merging aggregationprimitive
- parsing metrics taxonomytheory
- HDFS datasetdataset
- BGL datasetdataset