The Data Engine Room

Well, it looks like my predictions as to what Microsoft would do didn't happen. What is normally a quite release period given summer holidays has been pretty full on and we got an August release drop as well. Fabric platform New view in deployment pipelines Sadly this is about as much as we get in the CI/CD space for Fabric - and it doesn't address what is currently the weakest part of the platform in my opinion (I've been saying this for a while now). All this one does is add a new view to make it easier to select components across multiple folders. Great if you are an SME with a small enough team to make deployment pipelines work for you, but it doesn't address the complexities that come with enterprise grade deployments still. API specifications For those using the Fabric APIs, we now have a Github repository that contains all the specifications for the available APIs. Data Engineering Auto-scale billing for spark I know a number of people haven't been happy with...

 Just when I thought the product team might be slowing down for year end and the summer break, they've only gone against my expectations and not only dropped a July feature - but a couple of updates since. Given this one wasn't expected, I'm not going to be able to do my usual breakdown of what is GA and what's not - but I'll try to call it out as we go along. Up first we've got a fairly major update to the Spark compute - one that changes the billing approach Autoscale billing for spark (GA - Generally Available) For those that haven't seen it, this is a pretty big shift in the way that the spark compute is billed. Most of you are probably familiar with the SKU based approach that Fabric follows - that is that you bay for a particular license level and you get a number of compute units (think of it as credits) to spend on the activities you do in Fabric. The idea being that it gets around the difficulty of budgeting for the CFO that consumption based proces...

Last up in this month series is a review of the preview features we're seeing drop into both Fabric and Power BI. As always, do remember that it is not recommended to use preview features in production unless absolutely necessary. By using them in production you are carrying the risk of breaking changes, and without SLA level support. First up, Power BI. Power BI Visual calculation updates This month we've seen: Parameter pickers for required parameters when a function template is used. Coloured highlights similar to the way that Excel works. Change in behaviour when using columns in a function so that a value instead of an error is returned. Added visual calculations to the explore view. For me, the largest update in this change is adding the visual calculation option to the explore view. This feature will make it easier to allow train of thought analysis to move into areas not natively supported in the semantic model. Whilst the coloured highlights are welcome, my concern is ...

 As we approach the end of June, this months release notes have dropped. As with Fabric, expect the GA changes to be relatively light. We're start with the general section. General Sensitivity labels For those using sensitivity labels, do be aware that we're seeing to changes on how these are applied to make collaboration easier. The end result will be the same, but the label travels with the PBIX now.  If you are using sensitivity labels, make sure you check out the blog: Power BI June 2025 Feature Summary | Microsoft Power BI Blog | Microsoft Power BI Reporting Sparklines Sparklines within tables have now gone GA having been in preview since December 2021! It's insane that this has been a preview feature for so long, and is a clear example of why we need SLAs on preview features going GA across both Power BI and Fabric. If this is something that you've been wanting to use, but been able to because of company policy, it's now time that you can go in and start apply...

 As we head into July, we get what is likely to be our last major update until Fabcon Vienna. With holiday season and Microsoft's end of financial year, we don't typically see July and August releases, and I wouldn't expect any different this year. For those new to the blog, I post three separate blogs covering Fabric GA features, Power BI GA features, and preview features both combined. This one looks at the release notes that aren't clearly preview (though don't expect massive changes this month as we head into what are typically quieter times for Microsoft). First up is data engineering. Data engineering Notebook version history On the surface this one seems a minor change, but actually it's a massively handy feature. At one time or another, pretty much everyone makes a change that they need to undo but they haven't committed changes to version control as it's code they're still working on and isn't ready for a commit. Until now that meant, in...

Over the last few weeks I seem to be having more and more conversations about MCP servers. Be it with customers looking at agentic AI solutions, or announcements from the big tech providers.  With this in mind, I thought I'd write this blog to help engineers and architects understand what I've learnt about how they fit into the agentic solutions that data science teams are building today. What are MCP servers? An MCP, or model context protocol, server is ultimately an interpolation layer that makes it easier for data science teams to integrate APIs, data sets, etc into large language model (LLM) solutions.  Within these setups, an API, data set, etc is referred to as a tool. Historically, integrating one tool into an agentic solution has been relatively straight forward; but, that effort doesn't scale linearly. As more and more tools, the complexity rapidly increases - increasing both development and maintenance costs. To solve this challenge, Anthropic introduced MCP. This...