Lets take a look at a scenario I frequently use to explore what I mean by “regret“. How does it affect systems, our relationship with our customer and how it leads to a more complex system, despite our best efforts.
For this example, we’ll consider a very simple system that controls a door. One of those things we use to walk through, when it’s not a jar.
The two approaches
For this scenario, we’re going to consider two architectures. One entity obsessed and one using event sourced domain model.
We’ll assume our entity obsessed system is well architected with CQRS, featuring good separation between commands and queries with command handlers performing command execution; however commands and queries share the same underlying entity.
In our ESDM system, we’ll assume we have meta-data for things like when the event occurred, the identity that executed the command, the subject and source for the command etc etc. These are cross cutting concerns, not specific to the command; we don’t need a magic eight-ball to know we need these on every command and event emitted.
For both scenarios, we’ll ignore add commands for our door.
Things to look out for
We’re looking for changes across our system when a new requirement comes along. How many layers are updated? How many tests? How many commands? These changes can be explicit code changes, but can also be implicit change by increasing risk based on n-tier change to share components.
Requirement 1
Initially, the customer wants to be able to OPEN the door and CLOSE the door, and show the current state of the door.
Entity Obsessed System
We build the entity with an ID, and a boolean to indicate if it’s currently OPEN or CLOSED (let’s call it “OPEN”).
// Door Entity
{
id: guid,
open: bool,
created: date,
modified: date
}Because we’ve been hurt before, we add “Modified” and “Created” to the entity; even though no one asked for it. Because we know we’ll need it (from experience). The complexity begins!
Event Sourced Domain Model System
In our ESDM system, we have two commands; OpenDoor and CloseDoor. Each emits an event; DoorOpened and DoorClosed respectively. The commands are concerned only with the matter at hand; can the door be opened in its current state; can it be closed in the current state. No other considerations. Events are emitted into a retained event store. There is no entity storage to update.
The event payload for these events is actually empty. The wrapping event envelope contains the important information; the event type, the subject (the door, which I assume is the aggregate), the source (the command), the identity performing the command (the user) and when the event occurred.
{
subject: string,
source: string,
when: date,
securityContext: object,
type: string
}We have a projected view that shows the current state of the door; it’s ID and whether its open or not; the fact this looks like the entity above is very common – first views often like like traditional entities; though they only have a read only role and are not used (or even accessed) when executing future commands.
// Door View Projection
{
id: guid,
open: bool,
created: date,
modified: date
}Requirement 2
Now the customer has been using the system a little, they would now like to know what users have opened the door, and how often.
Entity Obsessed System
OK right. Let’s create an entity linked to our door entity via a foreign key. It records the name of the user, when they last used the door as a datetime and a count of how often they used it. We’ll need transactions to ensure we count the number of uses correctly without race conditions dropping counts or duplicating on partial invokes. This increases complexity of the system, requiring a storage migration and changes to the storage layer.
Areas in the diagram marked in pink have been changed (and therefore have increased complexity and risk).
We need to update our commands with more lines of code – OpenDoor and CloseDoor – so they both write to this new entity (table, probably). We need to update our query handlers and query result models with this new table. We need to update all our command handler tests with these changes. And we need to remember in our command layer why we’re doing all this so we don’t revert it naively in the future.
Complexity is starting to increase quadratically, and we’ve only just started!
Critically, we do NOT have this data for the current system state. Sorry, but that’s just how computer systems work, don’t you know dear customer. We can’t magic up data we weren’t collecting. You should have thought of this before, dear customer. This sucks, and erodes the customer relationship. This is regret for the customer and ourselves. But *shrug* this is software development.
// DoorUser entity
{
doorId: guid,
user: guid,
lastOpened: date,
openCount: int
}Its worth noting here that should this DoorUser table be destroyed through misadventure, all data (not backed up) is removed. Even if we did back it up, we need to kick off disaster recovery processes we’ve probably never yet tried.
Event Source Domain Model System
We don’t need to add any commands or events. We just add a new view projection that replays all existing events to render the required view model, as above. Because our read model is entirely separated (via logic abstraction in our assemblies) we have no changes in our write model (Commands) or event source. Complexity has increased in a linear way in our read model, and has not increase in our write model (commands).
// DoorUser View Projection
{
doorId: guid,
user: guid,
lastOpened: date,
openCount: int
}Critically, we can show all door access information from the beginning of the system. Our customer is happy, and confident in our supportive approach. We can reach back in time and access the data we did not know we’d need. This is minimising regret, making it easier for us to make independent decisions without fretting. It increases our credibility and our relationship with our customer.
Requirement 3
Now the customer would like to know all instances the door has been opened and closed by each user, so they can build an understanding of movement through the building.
Entity Obsessed System
OK… super. We now need to add a new DoorLog entity that records an individual open or close command. This in addition to the existing approach, as we still need door open counts available.
// DoorLog entity
{
doorId: guid,
user: guid,
opened: bool,
when: date
}Critically, once again the bloody customer has come up with a new idea that completely twatts all our hard work. We grumble and moan and point out that we cannot possibly provide this data from anything prior to this going into production – to expect otherwise would be madness. Do they not know how software works?? This further erodes relationship between engineering and the customer. This is regret for our customer and us; if only we’d known. Perhaps we should have spent time just thinking harder at the start, we could have predicted this requirement. We are such fools.
Complexity in the command layer continues to rise for OpenDoor and CloseDoor commands. We add services and storage interaction to be called inline within the command handlers to add the row to the DoorUserStateChanged table, which also adds complexity because it needs to be idempotent to ensure data integrity and deal with partial writes (since we’re writing to 3 tables now). We now have command handlers with around 100 lines. Query handlers and endpoints for the new queries need to be added.
It’s also worth noting this data is our source of truth, as before. If this table is lost through misadventure, we loose all the data.
Event Source Domain Model System
Sure thing boss! We already record identities with our events, which we have always retained. We add a new view projection that renders a list of user state changes for each door. We then simply replay the events from our aggregates to render the new view. We need to add the query handlers for the new view and add API endpoints for the new queries; our command (write) layer remains completely unchanged.
Critically, once again this data will include all door events from the start of the systems life; there is no regret here. No previous decision is causing the business to be hamstrung and to suck it up – and we’re building a great relationship with our customer by just saying yes.
Complexity has increased in a liner fashion, and only in the read model. Not a single line has changed in our write model (command layer).
And so on, etc
I could continue. Perhaps the customer now wants to have a report of door closed count (we recorded only openCount before). Perhaps the customer wants to know who is in the building right now for evacuations. Perhaps the customer wants to know how long people stay in the building. Perhaps perhaps perhaps.
Our bitter experience teaches us we should plan better, design better so none of these nasty surprises happen. Our complexity increases inconveniently over time – if only we just got it right at the start. This leads to wasteful over thinking and fretting at a time when we know the least and therefore are just guessing.
Change starts to take longer and longer because we a) have to think harder and b) we increase complexity and c) deploy larger, riskier changes.
This (I believe) is all driven by regret, through system architecture that accrues regret by being obsessed with the entity and it’s current state only.
Conclusion
Hopefully this illustrates with a pretty small, contrived example the regret we often form for ourselves, and how using ESDM is a way to minimise regret while allowing us to make small, independent decisions joyfully and with confidence, while building a great trusting relationship with our customers.
