Fabric September updates - My views
As we move forwards, I'm going to take a look at the monthly release notes and discuss the good, the bad, and the ugly.
First up:
Power BI
Desktop
First up, dark mode:
Image from Microsoft's release notes
Doesn't really need much explanation - apart from, about time! Only other major changes for desktop are a few formatting tweaks that, whilst are still in preview, are now turned on by default. Personally, I'd leave them on and roll with it.
Next, is how the interaction between Teams and Power BI works when MFA is enabled. This one had the potential to have been a right pain, but thankfully it only needs to be done once it seems.
For me, the biggest announcement is the live edit of semantic models running in Direct Lake mode from Power BI desktop. Up until now BI dev's working on a Direct Lake semantic model have had to work completely in the service - and I know this has annoyed a lot of people. This change of working practices was pretty counterintuitive. For me, being able to develop direct lake models in desktop should have come pretty quick after GA (if not at GA). I get why it wasn't part of MVP, but waiting this long has alienated a fair few people. Hopefully it isn't too late to win them back.
Service
When it comes to the service, we have a new object type called an org app now in preview. Basically it allows us to publish multiple apps from a single workspace. It seems a really promising feature that will make it easier to share apps across multiple domains; however, at the time of writing it seems a critical bug has been found and Microsoft aren't enabling it in any new tenants. For me, one to stay away of for a few months yet.
Next up, we have metrics sets. Thankfully the current metrics option (formally known as goals) is being renamed back to goals. From what I can tell of this new feature is basically OneLake data hub, but for measures. So whilst it isn't going to stop KPI proliferation, at least it'll make it easier to govern. Unfortunately it hasn't rolled out to my tenant, but I'll try to get some hands on time soon as I can.
Mobile
Something I wasn't expecting to see is NFC support within the app. The ability to add filters to the NFC tag is going to drive some really interesting use cases, and it is way more useful than the barcode scanning we previously had.
Fabric
Core
Previously Trusted workspace access and managed private endpoints have been F64 only features. Not anymore, they are now available for all trial and paid capacities. This is great news for those accessing other Azure items that are rightly secured behind a firewall (e.g. storage accounts, SQL DB, Event Hubs, etc).
Next, MTO Entra configurations are now supported - saving a massive headache for those in that scenario. You'll know if it's you, for everyone else it doesn't really matter.
Big news in this space is Git (Azure Dev Ops and Github) going GA. Yes it should have been GA from day one, and yes it is bad that we're not getting support for all items until the end of 2024, but at least it's only a couple more months to go on this one.
Next we have a new design for deployment pipelines. Having had hands on time with this during private preview, I can confirm it's a big step in the right direction. It's a lot less confusing than before. I've only had limited hands on with it early on during private preview, but I didn't find any bugs at that point. If you can afford to try it out, I'd recommend making the switch.
OneLake
First up is the mirroring of Azure Databricks Unity Catalogue into OneLake. Sounds great in practice, but my advice is run away as fast as you can from this one. For me, the fact that all queries fired down these shortcuts impersonate the user that setup the unity catalogue connection, and that Fabric can't be setup as a trusted service, it is too risky to even try right now. Yes you can secure it with workspace access, but it still opens up the possibility of individuals getting access to data they shouldn't have. I'm sure this will be resolved, hold tight, and don't get tempted.
Next we have shortcut support for GCP and AWS buckets - including over the on-premise gateway. For those running multi-cloud strategies it's definitely worth looking into.
On a theme of shortcuts, we now have GA availability of REST API support for create, list, and delete of shortcuts over API. Perfect use case for this is managing shortcuts as part of your ADO deployment pipelines. Combine this with the new Terraform features that are in public preview, and we're starting to see some pretty advanced use cases becoming possible.
Data warehouse
First up, copilot:
Next, we have SQL notebooks in the Warehouse. Great for those that are more familiar with the notebook format, but not going to be a major change for most.
Lastly, nested CTE support has gone into preview. Again, another feature that it's a bit shocking we've got this far without support. Hopefully it doesn't hang around in public preview too long.
Lakehouse
For the Lakehouse, we get new features on schema support. But don't worry about that, at the moment schema support is buggy, and personally I wouldn't use it until it goes GA.
Engineering
The big announcement in this space is the new capacity management options available at a workspace level. By combining setting maximum reserve cores, reserving maximum cores for active spark jobs, and setting timeouts for interactive notebooks, we can make sure our dev setup doesn't consume the entire capacity. It's a great start for adding guardrails on projects. It is in preview, but definitely something worth exploring.
Next, spark diagnostic emitter looks promising. It allows you to emit logs and metrics that can be consumed back into Fabric via real time event hubs. If this is of use to you, do go check the release notes for how to set this up.
Next we have notebook debugging via vscode. Once enabled it allows us to put in breakpoints and properly understand what is going on. Unfortunately it is in public preview, but anyone that has tried to debug notebooks before will know how big a win this is.
Image from Microsoft's release notes
Lastly, we have our first update to the runtime engine to v1.3. I'm not going to go into the release notes on that, and if you want to know more head over to https://learn.microsoft.com/en-us/fabric/data-engineering/runtime-1-3
Data science
I will confess, I'm not a data scientist and so a lot of this section doesn't mean as much as it could to me. From what I can tell we have a bunch of public preview stuff, but nothing major. The one thing I will call out is the public preview of AI skills.
I've had some hands on time with AI skills this week, and they do look promising. The joins that the skill used all made sense, but it did struggle with a central date table (I had to explicitly call it out in my prompt to get it to work properly). Personally, I think these need to be built over the semantic model rather than the Lakehouse. That way all the context that engineers have added (such as marking date tables, setting up relationships, etc) can be used and it would give a way for field/measure aliases to be added in the future (as happens with Q&A in Power BI desktop) and ultimately give Copilot way more context than a Lakehouse will. If anyone from Microsoft is reading this, please make it happen!!!
For everything else data science, do head over to the release notes.
Real time intelligence (RTI)
RTI has received quite a bit of love this month, the quick overview is:
- Copilot (again!) for dashboard creation and KQL translation
- New UI/UX changes
- New connectors for source and sink
- Data activator alerts on KQL query sets
The big new feature this month is the multivariate anomaly detection. This replaces the previous AI anomaly detector service. It'll be useful in a number of scenarios (such as fraud detection), and more details can be found at https://learn.microsoft.com/en-us/fabric/real-time-intelligence/multivariate-anomaly-overview.
Data factory
I know a lot of people use them, so it is fairly important that Dataflow Gen2 is now GA. Personally, for me, the issues around ownership are too much and I will carry on avoiding them like the plague.
The bigger news, especially those migrating to the cloud, is that on-premise gateway with pipeline support is now GA.
That's it for this month. Check in again, same time next month for more opinions on next months release.
Comments
Post a Comment