I have my data from AWS cloudera impala tables which receives data from IOT sensors (via OSI Soft PI data historian). I would like to connect this with Power BI along with SAP production order data to create a dashboard. IOT data gets collected every minute and sent to cloud. Hence the IOT data is massive. I have two questions
1 - I have noticed such massive data slowing down power BI dashboard. How to handle this slowness issue in data flows. What is the best architecture/data flow I should go far?
2 - Also I need specific access given to various levels of users as some of the data used in dashboard is confidential & meant for senior excutives only. How to set access security in data level itself? This is to prevent people from accessing the data source with self-service BI directly.
What are the different technical solutions possible for this scenario?