PowerBI is failing to load in services for few users, while other users are loading in seconds

I am seeing this peculiar issue, where few of my users keep seeing loading page.
While other users who are in same tenant, same capacity, same VPN and even same internet are able to load reports in seconds.

I have checked for any hardware issues as well, it is all fine.

Recommendations I tried:

1. Cleared all the cache in the browser
2. Tried Chrome, Edge etc
3. We also tried on other machine with same creds, still dont work
4. I have checked the size of the dataset in workspace it is ~30MB
5. RLS checked -- He is admin of it, he has read, reshare and write permission

   There are other users who are having same configuration and are able to use it within seconds.