Fabric - March 2025 release

 As part of my regular series of thoughts on the new release, this month is going to be slightly different given how many features we've had drop.

This blog is going to focus on new features that aren't part of Power BI, not in preview, and didn't make the keynote. If you want to see any of these, have a look here:

• Keynote
• Power BI
• Preview

Going forwards, I'm probably going to split out the Power BI updates from the Fabric updates. That way it allows me to go into key features in a bit more detail without ending up with a massive blog.

First up, general platform changes - and as always, don't forget to check the blog for more information on any of these features.

Fabric platform

CI/CD changes

Couple of important changes in this space. The main on is the APIs starting to support service principles. A welcome addition that removes the need for service accounts when managing CI/CD via ADO. Whilst the main feature we need of APi support when ADO is the git provider isn't present still, at least we've had confirmation that this is being worked on.

Other than that, we have a minor tweak to the UI to add branch out to an existing workspace instead of being forced to create a new workspace each time. This means that devs can have their own workspace that can be re-used as they move from one Jira/ADO ticket to another - reducing workspace scrawl. Anything to help that is always welcome!

Partner workloads

For specific use cases, partner workloads are now GA. For example, those with MDM challenges might lean on a native integration with Profisee and save a load of custom integration. Today the number of partners in the workload is pretty small but that'll only grow.

Certainly worth checking out to see if a native Commercially Off The Shelf (COTS) solution exists for your problem as part of your process to find a solution to your challenges.

Tags in Fabric

Tags have now gone GA. These are effectively metadata that helps to enhance searchability across the platform, making it easier for admins to manage the platform and users to find the insight they are looking for. The key to remember is it can take several items for a new tag shows up next to the item - but it should be searchable.

At the moment the fact we have to tag every item is adding a lot of overhead, and a half way house of some items being tagged and some not means that it probably isn't going to be that useful in reality. Personally I'd like to see default tags be able to be set at a workspace level - for example if I have a workspace that is a dev enviroment, let me tag the workspace so that all new items inherrit that tag. 

Fabric domains and OneLake Catalog UI changes

Personally, I'm finding more and more customer scenarios that a data mesh starts to make sense to push responsibility and accountability for data quality back down to the domain experts. 

This month we get better integration of OneLake catalogue, meaning that we can now filter by domain. This is a good extension for those using domains and subdomains, but personally I'd like to see domain settings reflected back into Purview.

If I've gone to the level of establishing these domains, let me filter by them in the Purview so I can do things like monitor the data quality in a specific domain, raise risks and audit actions against those accountable for a specific domain, etc.

Beyond that we have a number of UI enhancements to make it easier to find what users need and allow them to undertake common tasks quicker.

OneLake catalog and Modern get data are now integrated into Excel for Windows

Like a lot of BI developers, I have an ingrained dislike of Excel. The number of times that you deliver a wonderful report for it to be ignored and not replace the old Excel approach is always disheartning. It took me years to recognise that's a failure of BI programes. Either we're not doing the change management piece fully, or we're trying to enforce our recommendations rather than bring insight into end users workflows. 

The key message:

Don't try to kill Excel, learn to live along side it!

Given this, it's great to see improvements that make it easier for end users to enrich their Excel experience with data products from Fabric. It'll help bring the two worlds closer. You never know, some users might jump ship and start moving to Power BI instead.

Data Loss Prevention (DLP) policies for KQL and Mirrored DBs

For those using Purview DLP, the number of Fabric items it can apply to has been extended and new features added for lakehouses. If you use this feature jump into the blog for more info.

Multi-tenant organisation

This setup has now gone GA. For those that run in multi-tenant enviroments (such as consultancies), Fabric now supports Entra B2B - but with some restrictions.

if this is a setup you are likely to use, make sure that you jump into the documentation: https://learn.microsoft.com/en-us/power-bi/enterprise/service-admin-azure-ad-b2b 

OneLake

Clearly OneLake security is the big announcement in this space, but jump over to my keynote review if you want to see more on that.

Other than that, the announcement is that OneLake shared access signitures (SAS) are now GA. Basically these allow you to share data stored in one lake securely via Entra ID and for up to 1-hour. I suspect this will mostly be for ISV integration, but some people could have use cases that this would be helpful for.

Data engineering

Write capabilities and PySpark support in Spark Connector for Fabric DW

For those running an enviroment that's a mixture of Lakehouses and Warehouses, it's now easier to move data between the two whilst applying transformations - without suing workarounds.

Row and Column level security in Spark

Yet another way we can control access to our underlying data. Personally, my view is if you can wait for OneLake security to beomce more feature rich then I would do that. otherwise, this might provide a good stop gap until OneLake secuirty is more richj.

Either way, if you use this, keep it on the backlog that you'll probably see this as long term strategic debt that will need to be repaid later.

Introducing Pylance language support for Fabric Notebooks

From what I can see the best way to think of this is: improved auto-completion, enhanced Lambda expresiion support, and better documentation and UI improvements.

The one thing that isn't clear is if this is still in preview or if it is now GA. The learn documentation suggests it's in preview, but the release notes aren't tagged as such. 

Personally, I'd use it but with a grain of salt that it could carry the usual risks of preview features.

Shortcuts now supported in Lakehouse Git metadata reporesentation and in Fabric Deployment pipelines

Another feature that seems to be GA in the release notes but Learn is tagged as a preview feature.

Again, given that Git integration of all components should have been MVP, I would say the risk of using this preview feature vs loosing code is worth the risk. But do be aware that you might run into some challenges if the Learn documentation is right.

Data science

Copilot in Notebooks, Agentic, and UX enhancements

First up is several changes to copilot in notebooks. In the same way as the Power BI changes, Copilot in notebooks now has:

• Enhanced conversation history
• Enhancements to chat and natural language output
• Improved code generation capabilities
• New Quick Actions button to simplify common tasks

For those that missed it, AI skills have also been renamed as Fabric data agents. Why? To better align with their true use cases and make them easier to understand at face value.

Beyond this is another call out to the availability of copilot across all SKUs. Make sure you check out the preview blog for more data science features.

Data warehouse

A lot of this months features are only private or public previews. Beyond those, it's focused on UX improvements:

• Intellisense for collate clause
• Intellisense improvements for JSON data in OPENROWSET
• Live templates in intellisense to improve development efficiency
• Improvements to search and filter experiences
• Artifact status bar
• Cancel query option on closing editor
• Show keyboard shortcuts

Beyond this, we have enhanced sharing capabilities to grant permissions to monitor queries and audit activities.

Real-Time intelligence (RTI)

New eventstream sources

This month we've had eventstream sources for:

• MQTT connector
• Solace PubSub+
• Azure Data Explorer
• Real-time weather
• Azure Event Grid

Eventstream CI/CD & Rest APIs go GA

For those using eventstream Git integration and using release pipelines, thease features are now in GA removing the risk you'd been carrying so far. The same goes for the Eventstream rest API

Expanded availability for Outband Event stream private endpoints

For those that aren't using it, this means we can place our outbound eventstreams inside a managed vNet and still send the data out of the eventstream and into a PaaS managed Azure event hub.

Connect to Eventstream using Microsoft Entra ID authentication

For those using custom endpoints for inbound streams, we can now use Entra ID to configure the access on our custom endpoint.

Improvements to eventhouse get data experience

We've had a few improvements to the UX when getting data into an Eventhouse. These are:

1. Automated schema optimisation. This will pickout differences between source and taget schemas, and present recommendations
2. Simplified schema inspection. Making it easier to undertake a number of common steps in schema mapping
3. Real-Time data sampling to check that sink data will be correct before applying the changes
4. Automatic detection of header row for csv files

Build event-driven workflows with Azure and Fabric events

The subscribe to functions in the RTI hub within Fabric are now GA> Making it easier to establish event-driven architectures.

Eventhouse OneLake availability now supports backfill

Previously when you turned on replication from an Eventhouse to OneLake, only new data was streamed. Now, turning this on will backfil the historic data as well.

Improved activator alers from Power BI

Instead of having to manage alerts within data activator, these can now be managed within the Power BI report itself.

Data Factory

Connector changes

First up we have new/improved connectors for:

• Lakehouses
• ADP analytics
• Dynatrace Grail DQL
• Anaplan
• Asana
• BQE core
• BuildingConnected
• Delta Sharing
• SolarWinds service desk
• Supermetrics
• Windsor
• Worksplace Analutics
• Zendesk data

If you need any of these connectors, please make sure you check out the release notes.

Copy job goes GA

For those using a medallion architecture being able to use a direct copy into bronze will help improve performance. Now that this has hit GA, it becomes an option.

Beyond this, the copy job has gained:

• public API and CICD support
• Upsert functionality
• Improvements to the data mapping workflow
• Real-time monitoring with an in-progress view
• Customisable update menthods & schedules before job creation
• Increased connectors 

CI/CD now GA

Whilst pipelines have had CI/CD capabilities for a while, it is now GA. Alongside this we also have CI/CD support for mirrored databases going GA too.

Connection parameterisation

Parameters can now be used as the connection. Allowing developers do things such as switch connections based on the enviroment they are deployed in.

Dataflow Gen2 table name parameter support for data detinations

Should you need it, you can now parameterise your destination names in Dataflow Gen2.

Copilot for data factory enhancements

Various enhancements have been made to copilot to make it easier to create, understand, and troubleshoot pipelines.

Orchestration

Apache Airflow job

The native integration of Apache Airflow into Fabric is now GA. Part of this process include enhancements to:

• Add a Fabric specific versioning. 1.0 is using Apache Airflow 2.10.4 and Python 3.
• Public APIs
• Git integration and deployment pipeline support
• Diagnostic logs

OneLake file triggers for pipelines now GA

The ability to run files when events such as FileCreated, FileDeleted, etc is now GA. Meaning that we can now build items such as drop boxes for loading adhoc data feeds.

Spark Job Definition pipeline and Azure Databricks job activity parameter support

For those using spark jobs in your Lakehouses, you can now use parameters when invoking your jobs from a Fabric pipeline. The same is true for Azure Databricks jobs.

Increase in number of activities per pipeline

Previously the number of activities in a Data Factory pipeline was limited to 80 activities. This has now been lifted to 120 activities.

Dataflow Gen2

You can now call Gen2 dataflows within pipelines.