The Data Engine Room

  Less than an hour after this mornings keynote and I thought I'd put together some initial thoughts on what we heard at Fabcon this morning. After some initial fun with dancing robots, and introduction from the team at Guy In a Cube, it was time for Arun to take to the stake and open the keynote. As we've come to expect from Microsoft keynote's over the last 12 months, the key theme was that of AI. But critically recognising how crucial getting your data estate in line is to taking advantage of the new developments we're seeing in technology today. As all us data experts know, the quality of the insight the business will obtain is only as good as the quality of the data that underpins it. Typically referred to as garbage in / garbage out (GiGo). Combine this with the fragmentation in the technology landscape, and businesses struggle to solve the dual challenges of interopobility and Gigo. A challenge that Microsoft Fabric is trying to solve today. For those that aren...

Some of you will be familiar with dynamic row level security (DRLS) in PowerBI. For those that aren't, DRLS allows us to use the email address of the user that is logged into the service via their Entra account. With this email address, and corresponding email address in our semantic model, we can dynamically filter the semantic model to only show the data that user could see. The result, we can use one report suite to serve the same content to country, region, and global teams whilst simultaneously providing the necessary security around sensitive data. Now back in the world of Power BI, that was really easy. We fired up Power BI desktop, went into manage security roles, and added the necessary DAX: Now in Fabric, that still works exactly the same way as it always has for import and DirectQuery semantic models. However, it's not so straight forward for DirectLake semantic models. Why? The challenge is that a number of features break DirectLake connections and force the model b...

Since I first heard about Microsoft Fabric during private preview, I have spent many hours pondering the workspace topologies available to us. With this workspace based setup it gives us the power to spin up workspaces quickly and easily. But with that power comes great responsibility. But why is that? As time goes on I can see us struggling from workspace proliferation in exactly the same way that BI has suffered from KPI and report proliferation. That is multiple copies of the same workspace but designed for slightly different purposes. To counter that we need to carefully plan and curate our workspace creation, meaning that we need a set of topologies that we can follow to make our lives easier. But what is a workspace topology? The best way to think of it is a blueprint for how we're going to create workspaces and link them together. As I thought about this more and more, three clear topologies began to emerge. I've called these: Monolith warehouse Hub and Spoke Data Mesh M...

As we move ever closer to Fabcon Europe and the speaker sessions get announced more and more, it becomes even more important to plan your few days . With so much crammed in to so few days, it's all to easy to miss a critical session. With that in mind, here's some of recommendations day by day of sessions I have on my 'must see' list. Tuesday If you've never had hands on time with Microsoft Fabric, get yourself on to the Fabric in a day session. This will take you from knowing nothing to building your first Fabric solution before the day's out. For those who have tried Fabric, it will very much depend on your specialism. Personally, I'd be heading to the professional software development on the fabric platform session. It touches some of the more advanced concepts that not many people will have been exposed to yet. Wednesday AM The keynote is a must. I'm sure we'll see some great product announcements, along with some surprises that aren't on t...

Earlier in the year I wrote a LinkedIn post about semantic models. To start this blog, I thought I'd expand upon the post and help those moving from an old school on-premise setup to a modern data platform such as Fabric. For the purposes of this post I'm going to focus on Microsoft houses, but the concepts being discussed are equally relevant for all BI platforms.

Well, it's happening at last. It's time to start to turn all the knowledge over the years back into something useful for the community. So, what can you expect from this site? My aim is to build it into something helpful for all data engineers. Initially I'll focus on news, views, and opinions from the Microsoft space. I'll cover what new features mean in reality, provide my personal opinion of what I can talk about and the my view on what others are saying, and I'll share the best practices that I'm seeing developing. Initially, the site will be very heavily focused on Microsoft Fabric (especially with Fabcon Europe coming up). But do expect some guest posts appearing as time goes on. Onwards and upwards.