Hello,
We have a connector that gets data from our server, and have RangeStart and RangeEnd and pagination support - All works great.
But, one of the column in the data has value which is a list of objects that are in this structure: [{"key": "column 1", "value": "value 1"}, {"key": "column 2", "value": "value 2"]
and we wish to flatten this list of object based on the keys.
each key - is a column. each value - will be its value in that row.
in the code - I've Pivoted this column successfully. and on Power BI Desktop - all works great.
but for each rows it can have different values for the key - so the columns at each page can be different.
so this is a legal options:
[{ "ID": 1, list: [{"key": "column 1", "value": "value 1"}, {"key": "column 2", "value": "value 2"]}, ... (several pages)
{ "ID": 2000 list: [{"key": "column 1", "value": "value 2"}, {"key": "column 3", "value": "value 3"]}, .... (several pages)
{ "ID": 330000, list: [{"key": "column 4", "value": "value 34"}, {"key": "column 35", "value": "value 46"]}]
after Pivot in code I got:
| ID | column 1 | column 2 | column 3 | column 4 | column 35 |
| 1 | value 1 | value 2 | |||
| 2 | value 2 | value 3 | |||
| 3 | value 34 | value 46 |
there could be many pages, many rows, severals columns names for each object (not only 2) could be none, and values can be nullables
as a result in the power desktop - when I refresh the model - the keys displays as columns - works great.
But- when I tried to set Incremental Refresh and publish it to Power BI Cloud - on first page there was an error say something like "The 'column 35' column does not exist in the rowset."
I guess this is something relate to the Folding requirement of the Incremental Refresh.
because Columns in the first page can be different then in the second one or in the page number 100... and the Power BI Incremental Refresh mechanism doesn't just "Add column" and append the data according to the name.
in Power BI Desktop it does.
What can I do? how can I Pivot the list in the connector code so our customer will easier get the data as a regular column and not need to make thier own manipulation like: Pivoting / Expanding ?
Please Advice thanks,
Liran