概述
继续跟中华石杉老师学习ES,第48篇
课程地址: https://www.roncoo.com/view/55
官方说明
Percentiles Aggregation: 戳这里
示例
需求: 网站访问时延统计
为了演示该功能,我们先建个索引,模拟一批数据
#新建索引
PUT /website
{
"mappings": {
"logs": {
"properties": {
"latency": {
"type": "long"
},
"province": {
"type": "keyword"
},
"timestamp": {
"type": "date"
}
}
}
}
}
#模拟数据
POST /website/logs/_bulk
{ "index": {}}
{ "latency" : 105, "province" : "江苏", "timestamp" : "2018-10-28" }
{ "index": {}}
{ "latency" : 83, "province" : "江苏", "timestamp" : "2018-10-29" }
{ "index": {}}
{ "latency" : 92, "province" : "江苏", "timestamp" : "2018-10-29" }
{ "index": {}}
{ "latency" : 112, "province" : "江苏", "timestamp" : "2018-10-28" }
{ "index": {}}
{ "latency" : 68, "province" : "江苏", "timestamp" : "2018-10-28" }
{ "index": {}}
{ "latency" : 76, "province" : "江苏", "timestamp" : "2018-10-29" }
{ "index": {}}
{ "latency" : 101, "province" : "新疆", "timestamp" : "2018-10-28" }
{ "index": {}}
{ "latency" : 275, "province" : "新疆", "timestamp" : "2018-10-29" }
{ "index": {}}
{ "latency" : 166, "province" : "新疆", "timestamp" : "2018-10-29" }
{ "index": {}}
{ "latency" : 654, "province" : "新疆", "timestamp" : "2018-10-28" }
{ "index": {}}
{ "latency" : 389, "province" : "新疆", "timestamp" : "2018-10-28" }
{ "index": {}}
{ "latency" : 302, "province" : "新疆", "timestamp" : "2018-10-29" }
DSL:
#网站访问时延统计(50% 95% 99%的请求)
GET /website/logs/_search
{
"size": 0,
"aggs": {
"latency_percentiles": {
"percentiles": {
"field": "latency",
"percents": [
50,
95,
99
]
}
},
"latency_avg": {
"avg": {
"field": "latency"
}
}
}
}
返回:
{
"took": 3,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 12,
"max_score": 0,
"hits": []
},
"aggregations": {
"latency_avg": {
"value": 201.91666666666666
},
"latency_percentiles": {
"values": {
"50.0": 108.5,
"95.0": 627.4999999999997,
"99.0": 654
}
}
}
}
可知,
- 50%的请求 在 108.5毫秒
- 95%的请求 在 627.5毫秒
- 99%的请求 在 654毫秒
- 平均时延 201.9毫秒
数值的最大的值是多少,不是完全准确的
需求二:按地区分组后看下每个地区的 网站访问时延统计
GET /website/_search
{
"size": 0,
"aggs": {
"group_by_province": {
"terms": {
"field": "province"
},
"aggs": {
"latency_percentiles": {
"percentiles": {
"field": "latency",
"percents": [
50,
95,
99
]
}
},
"latency_avg": {
"avg": {
"field": "latency"
}
}
}
}
}
}
返回:
Percentiles优化 compression
compression 官方指导: 戳这里
TDigest算法,用很多节点来执行百分比的计算,近似估计,有误差,节点越多,越精准
compression ,限制节点数量最多 compression * 20 = 2000个node去计算 ,默认100
compression 越大,占用内存越多,越精准,性能越差
一个节点占用32字节,100 * 20 * 32 = 64KB
简单来说: 如果你想要percentile算法越精准,compression可以设置的越大
本文标题:白话Elasticsearch48-深入聚合数据分析之 Percentiles Aggregation-percentiles百分比算法以及网站访问时延统计及Percentiles优化
本文链接:https://blog.quwenai.cn/post/6356.html
版权声明:本文不使用任何协议授权,您可以任何形式自由转载或使用。












还没有评论,来说两句吧...