将 Vega 与包含嵌套聚合的 Elasticsearch 数据一起使用或在 Elasticsearch 中将一个聚合除以另一个

如何解决将 Vega 与包含嵌套聚合的 Elasticsearch 数据一起使用或在 Elasticsearch 中将一个聚合除以另一个

我正在尝试使用 Elasticsearch 做一些应该非常简单的事情。我有一个索引，其中包含以下形状的文档：{"timestamp": int,"pricePerUnit": int,"units": int}。我想在直方图中可视化一段时间内的平均价格。请注意，我不想要“pricePerUnit”的平均值，我想要每个单位支付的平均价格，这意味着通过将每个文档的“pricePerUnit”乘以“units”来找到每个时间段中的总价值，并且将每个文档中销售的总价值相加，然后除以时间段中销售的总单位数的总和，以获得每单位支付的平均价格。标准的 Kibana 折线图不起作用。我可以获得平均“pricePerUnit * units”，但不能将此聚合除以总单位的总和。也不能在 TSVB 中完成，因为这不允许脚本/脚本字段。不能使用 timelion，因为“时间戳”字段不是时间字段（我知道，但我无能为力）。因此，我正在尝试使用 Vega。但是，我遇到了嵌套聚合的问题。这是我正在运行的 ES 查询：

{
  "$schema": "https://vega.github.io/schema/vega/v3.json","data": {
  "name": "vals","url": {
      "index": "index_name","body": {
  "aggs": {
    "2": {
      "histogram": {
        "field": "timestamp","interval": 2000,"min_doc_count": 1
      },"aggs": {
        "1": {
          "avg": {
            "field": "pricePerUnit","script": {
              "inline": "doc['pricePerUnit'].value * doc['units'].value","lang": "painless"
            }
          }
        }
      }
    }
  },"size": 0,"stored_fields": [
    "*"
  ],"script_fields": {
    "spend": {
      "script": {
        "source": "doc['pricePerUnit'].value * doc['units'].value","lang": "painless"
      }
    }
  },"docvalue_fields": [],"_source": {
    "excludes": []
  },"query": {
    "bool": {
      "must": [],"filter": [
        {
          "match_all": {}
        },{
          "range": {
            "timeslot.startTime": {
              "gte": 1621292400,"lt": 1621428349
            }
          }
        }
      ],"should": [],"must_not": []
    }
  }
},"format": {"property": "aggregations.2.buckets"}
    }
    },"scales": [
    {
      "name": "yscale","type": "linear","zero": true,"domain": {"data": "vals","field": "1.value"},"range": "height"
    },{
      "name": "xscale","type": "time","range": "width"
    }
  ],"axes": [
    {"scale": "yscale","orient": "left"},{"scale": "xscale","orient": "bottom"}
  ],"marks": [
    {
      "type": "line","encode": {
        "update": {
          "x": {"scale": "xscale","field": "key"},"y": {"scale": "yscale","field": "1.value"}
        }
      }
    }
  ]
}

它给了我以下结果集：

  "took": 1,"timed_out": false,"_shards": {
    "total": 4,"successful": 4,"skipped": 0,"failed": 0
  },"hits": {
    "total": 401,"max_score": null,"hits": []
  },"aggregations": {
    "2": {
      "buckets": [
        {
          "1": {
            "value": 86340
          },"key": 1621316000,"doc_count": 7
        },{
          "1": {
            "value": 231592.92307692306
          },"key": 1621318000,"doc_count": 13
        },{
          "1": {
            "value": 450529.23529411765
          },"key": 1621320000,"doc_count": 17
        },{
          "1": {
            "value": 956080.0555555555
          },"key": 1621322000,"doc_count": 18
        },{
          "1": {
            "value": 1199865.5714285714
          },"key": 1621324000,"doc_count": 14
        },{
          "1": {
            "value": 875300.7368421053
          },"key": 1621326000,"doc_count": 19
        },{
          "1": {
            "value": 926738.8
          },"key": 1621328000,"doc_count": 20
        },{
          "1": {
            "value": 3239475.3333333335
          },"key": 1621330000,{
          "1": {
            "value": 3798063.714285714
          },"key": 1621332000,"doc_count": 21
        },{
          "1": {
            "value": 482089.5
          },"key": 1621334000,"doc_count": 4
        },{
          "1": {
            "value": 222952.33333333334
          },"key": 1621336000,"doc_count": 12
        },{
          "1": {
            "value": 742225.75
          },"key": 1621338000,"doc_count": 8
        },{
          "1": {
            "value": 204203.25
          },"key": 1621340000,{
          "1": {
            "value": 294886
          },"key": 1621342000,{
          "1": {
            "value": 284393.75
          },"key": 1621344000,{
          "1": {
            "value": 462800.5
          },"key": 1621346000,{
          "1": {
            "value": 233321.2
          },"key": 1621348000,"doc_count": 5
        },{
          "1": {
            "value": 436757.8
          },"key": 1621350000,{
          "1": {
            "value": 4569021
          },"key": 1621352000,"doc_count": 1
        },{
          "1": {
            "value": 368489.5
          },"key": 1621354000,{
          "1": {
            "value": 208359.4
          },"key": 1621356000,{
          "1": {
            "value": 7827146.375
          },"key": 1621358000,{
          "1": {
            "value": 63873.5
          },"key": 1621360000,"doc_count": 6
        },{
          "1": {
            "value": 21300
          },"key": 1621364000,{
          "1": {
            "value": 138500
          },"key": 1621366000,"doc_count": 2
        },{
          "1": {
            "value": 5872400
          },"key": 1621372000,{
          "1": {
            "value": 720200
          },"key": 1621374000,{
          "1": {
            "value": 208634.33333333334
          },"key": 1621402000,"doc_count": 3
        },{
          "1": {
            "value": 306248.5
          },"key": 1621404000,"doc_count": 10
        },{
          "1": {
            "value": 328983.77777777775
          },"key": 1621406000,{
          "1": {
            "value": 1081724
          },"key": 1621408000,{
          "1": {
            "value": 2451076.785714286
          },"key": 1621410000,{
          "1": {
            "value": 1952910.2857142857
          },"key": 1621412000,{
          "1": {
            "value": 2294818.1875
          },"key": 1621414000,"doc_count": 16
        },{
          "1": {
            "value": 2841910.388888889
          },"key": 1621416000,{
          "1": {
            "value": 2401278.9523809524
          },"key": 1621418000,{
          "1": {
            "value": 4311845.4
          },"key": 1621420000,{
          "1": {
            "value": 617102.5333333333
          },"key": 1621422000,"doc_count": 15
        },{
          "1": {
            "value": 590469.7142857143
          },"key": 1621424000,{
          "1": {
            "value": 391918.85714285716
          },"key": 1621426000,{
          "1": {
            "value": 202163.66666666666
          },"key": 1621428000,"doc_count": 3
        }
      ]
    }
  }
}

问题是我无法从“1”子聚合中提取“value”字段。我试过使用展平变换，但它似乎不起作用。如果有人可以：

a) 告诉我如何用 Vega 解决这个特定问题；要么 b) 告诉我解决我原来问题的另一种方法

我将不胜感激！

解决方法

您的 DSL 查询看起来很棒。如果我没看错，我相信您正在寻找的是 project transform。在处理嵌套变量时，这可以让生活变得更轻松，因为在某些情况下，它们无法按预期运行。

您还需要引用标记内的数据，否则它不会绘制任何内容。

以下是解决此问题的方法，您只需要在其中添加 url 参数即可。

{
  $schema: https://vega.github.io/schema/vega/v3.json
  data: [
    {
      name: vals
      url: ... // fill this in
      transform: [
        {
          type: project
          fields: [
            1.value
            doc_count
            key
          ]
          as: [
            val
            doc_count
            key
          ]
        }
      ]
    }
  ]
  scales: [
    {
      name: yscale
      type: linear
      zero: true
      domain: {
        data: vals
        field: val
      }
      range: height
    }
    {
      name: xscale
      type: time
      domain: {
        data: vals
        field: key
      }
      range: width
    }
  ]
  axes: [
    {
      scale: yscale
      orient: left
    }
    {
      scale: xscale
      orient: bottom
    }
  ]
  marks: [
    {
      type: line
      from: {
        data: vals
      }
      encode: {
        update: {
          x: {
            scale: xscale
            field: key
          }
          y: {
            scale: yscale
            field: val
          }
        }
      }
    }
  ]
}

以后如果您遇到问题，请查看 Vega Gallery 上的示例。他们也有extensive documentation。这两个组合就是你所需要的。