Im trying to incrementally append data in Lakehouse using DataFlow (as described here: https://learn.microsoft.com/en-us/fabric/data-factory/tutorial-setup-incremental-refresh-with-dataflows-gen2)
I did everything in the same way as described.
If I open DataFlow I can see in preview that it works as my main query/table is filtered according to the filter based on maximum order_id (the result of the second query).
The problem is when I want to refresh the dataflow. I get an error message "We cannot use operator < for Table and Number types"
Any idea what is wrong? I guess that if I can see in preview that the table is filtered according to maximum order_id so data type is ok (number). The filter is set exactly as in the example described on LearnMicrosoft site.
Thank you
The whole text of error message:
Error Code: Mashup Exception Expression Error, Error Details: Couldn't refresh the entity because of an issue with the mashup document MashupException.Error: Cannot insert table., Inner exception: We cannot use operator < for Table and Number types., basic error: We cannot use operator < for Table and Number types. Details: Reason = Expression.Error;Message = We cannot use the < operator for Table and Number types.;Detail = [Operator = "<", Left = error "Microsoft.Mashup.Engine1.Runtime.ValueException: [Expression.Error] No value specified.#(cr)#(lf) at Microsoft.Mashup.Engine1.Language.ValueCreator.CreateValueForThrow(IThrowExpression throwExpr)#(cr)#(lf) at Microsoft.Mashup.Engine1.Language.ValueCreator.<>c__DisplayClass23_0.<CreateValueForRecord>b__0(Int32 index)#(cr)#(lf) at Microsoft.Mashup.Engine1.Runtime.RecordValue.DemandRecordValue.get_Item(Int32 index)#(cr)#(lf) at Microsoft.Data.Mashup.ProviderCommon.MashupResource.TryGetValue(Func`1 getValue, IValue& value, String& errorMessage)#(cr)#(lf)Record", Right = 2619 65];Message.Format = We cannot use the #{0} operator for types #{1} and #{2}.;Message.Parameters = {"<", "Table", "Number"};Microsoft.Data.Mashup.Error.Context = User GatewayObjectId: 4c147d88-9572-4e34-8a5b-4fe1391a873e (Request ID: 1318301e-5b5c-44a4-9ea9-30d02261f7e1).