Why is it important to understand?
When considering the Dynamics 365 Cross-Apps solution design, it is imperative to understand the internal dynamics of each system. Each system has its own capabilities and limitations.
And when it comes to transaction commitment to the database, each system has its own transaction tracking mechanism. Dynamics 365 Finance & Operations and Dataverse are not exceptions to this rule.
Transaction tracking mechanisms exist to ensure data integrity. I'm sure you have used either application before.
With Dynamics 365 FO, every transaction is part of the transaction scope, and a single transaction scope can include multiple changes. Sometimes it will be a complex case based on your business scenario.
As a result, it becomes the record event of table inserts and updates.
As you are aware, each transaction is being handled as a single UnitOfWork in Dynamics 365 FO to ensure data referential integrity.
What is UnitOfWork?
Think of it like this: You give a series of specific individual rows to a UnitOfWork object, in the form of table buffer variables. The UnitOfWork object successfully processes each row, or it rejects all changes. UnitOfWork patterns determine the sequence of delete, insert, and update operations on tables linked by foreign keys automatically.
This pattern was first introduced in Dynamics AX 2012 to manage database transactions. It automatically propagates the primary key value to the corresponding foreign key field, when the row with the foreign key field is inserted.
More information: https://learn.microsoft.com/en-us/dynamicsax-2012/developer/how-to-use-the-unitofwork-class-to-manage-database-transactions
But when you enable Dual-write table maps, live transactions data flows from Dynamics 365 FO to Dataverse and vice versa automatically.
When Dual-write is enabled for a table map, the "ttsprecommit” event is triggered before transactions are committed. This event only occurs for Dual-write enabled entities.
Success transaction flow
The above sequence diagram shows a representation of success transaction flow.
As you can see, there are three table’s transactions processing is going in Dynamics 365 FO.
And it finds out all sorts of modifications within the transactions, bundles them together and sends out a single batch request to Dataverse.
This messaging is happening over the wire including network latency, data volume and in a synchronized way.
It means unless the message comes back with the status of success or Failure, it will keep on waiting for the transaction to be committed in Dynamics 365 FO.
When a single batch request sends out to Dataverse then Dataverse initiates its own transaction scope. It automatically identifies what all tables are to be modified and these are all Dataverse tables and not Dynamics 365 FO tables.
It does all transactions at Dataverse side and if it is a success then sends a "Success" response code.
Dual-write runtime understands that response code. And it let Dynamics 365 FO go-ahead and commit the transactions on Dynamics 365 FO.
In terms of semantics, it is a very tightly coupled system due to the synchronised nature of live sync. The run-time for the dual-write is waiting for a response from the Dataverse side before it can complete.
It is a distributed transaction, since Dynamics 365 FO and Dataverse both have their own scope for transactions, and Dual-Write Runtime is trying to match both in a single process. As a result, transaction consistency is guaranteed.
Note: In distributed transactions, data is processed across multiple data repositories (most commonly databases).
Failure transaction flow
The above sequence diagram shows a representation of failure transaction flow.
In the event of failure, it sends the "Failure" response back to Dynamics 365 FO or Dataverse.
The failure may be caused by a business validation failure at Dataverse, resulting in Failure and Dual-Write run-time aborting the transaction.
At the pre-commit stage in the source, most of the validations are done on the source side and anything after that is basically the final commit activity from Dynamics 365 FO and if it is aborted then it will roll back the whole transaction.
How should design be approached?
Multiple records are inserted into one or more tables as part of a single transaction scope, such as the invoice posting routine.
As we learned earlier, all transactions of a single transaction block bundle get combined and committed as a single batch.
There could be multiple transaction blocks where a certain process could trigger multiple transaction blocks such as posting of invoices etc.
And each transaction acts as a single message sent back and forth between D365 FO and Dataverse.
So, each transaction must send one request to the other side for the commitment.
First aspect is complex business process
If due to some business process, lots of transactions are generated in your source system then it will bubble up to lots of network’s calls flowing back and forth because of dual-write.
When we are talking about the number of transactions on each side especially, we are referring to the UnitOfWork. The number of transactions would impact the number of calls made by Dual-write to another side.
A second aspect is the number of records per transaction
Each transaction is separate and, as a result, it will be able to commit on its own and will not be dependent on other transactions. However, each transaction can have many records, it could be 10 or 1000 or more than 1000.
There is just something complex about your business process. So, if somebody posts 1000 invoices, then there would be at least 1000 records in the transaction scope.
In other words, one transaction scope equals one transaction call.
All these records bundle up and pack together and send to another side as a single transaction.
I would highly recommend to understand Dual-write sync-limits and Service protection API limits from Dynamics 365 FO to Dataverse before designing the tables maps to avoid unnecessary performance issues.
There are also limits apply when data is written from Dataverse to Dynamics 365 FO.
More information: https://learn.microsoft.com/en-us/dynamics365/fin-ops-core/dev-itpro/data-entities/dual-write/sync-limits
What’s new coming in Dual-write?
A new feature will be available as a public preview on January 1, 2023. Additionally, dual-write functionality will support asynchronous, continuous data movement between Dynamics 365 FO and Dataverse in order to achieve data consistency in the future. As an alternative to Data Integrator, you can think of it as a tool for integrating data.
It will be possible for asynchronous business processes to participate Dual-write executing bulk create and update operations in asynchronous mode. Additionally, this feature will prevent session timeouts.
I hope this article was a good read for you. Do share it with your friends and other peers.
Thank you for Reading - Let's Connect!
Enjoy my blog? For more such awesome blog articles - follow, subscribe and let's connect on LinkedIn, Twitter, YouTube
Stay tuned!
