We are receiving the following error when Power Bi Service tries to refresh our dataset with the query below.
[Unable to combine data] Section1/Query1/AutoRemovedColumns1 is accessing data sources that have privacy levels which cannot be used together. Please rebuild this data combination.
let
// The FirstQuery return a list of records and a scrollId
FirstQuery = Json.Document(Web.Contents("https://<url to elasticsearch cluster>.com", [Query=[scroll="5m", size="10"]])),
// The SecondQuery uses the scrollId from FirstQuery to make its call
SecondQuery = Json.Document(Web.Contents("https://<url to elasticsearch cluster>.com", [Query=[scroll_id=FirstQuery[_scroll_id]]])),
// The list of records from FirstQuery and SecondQuery are combined and returned.
CombinedResults = FirstQuery[hits][hits] & SecondQuery[hits][hits]
in
CombinedResults
As you can see the query makes two requests to an Elasticsearch cluster and combines their response data. The second query uses the response from the first query as a query param. The authentication is done with Basic Auth and we use Power BI's native Basic Auth to authenticate the data source.
The query works fine locally using Power BI Desktop, it just fails when refrehsing on Power BI service.
This looks like a bug as AutoRemovedColumns1 is not part of our query. We have tried all different combinations of privacy settings Power BI offers, but nothing works. The only way we can get it to work is to hard-code the basic auth header into the web calls themselves and set the Authentication for the query within Power BI to Anonymous.
Does anyone have any suggestions, or is this a bug that Microsoft need to look into?