Introduction to Microservices

Barry S. Stahl

Solution Architect & Developer

Transparent Half Width Image.png

Favorite Physicists & Mathematicians

Favorite Physicists

  1. Harold "Hal" Stahl
  2. Carl Sagan
  3. Richard Feynman
  4. Marie Curie
  5. Nikola Tesla
  6. Albert Einstein
  7. Neil Degrasse Tyson
  8. Niels Bohr
  9. Galileo Galilei
  10. Michael Faraday

Other notables: Stephen Hawking, Edwin Hubble

Favorite Mathematicians

  1. Ada Lovelace
  2. Alan Turing
  3. Johannes Kepler
  4. Rene Descartes
  5. Isaac Newton
  6. Leonardo Fibonacci
  7. George Boole
  8. Blaise Pascal
  9. Johann Gauss
  10. Grace Hopper

Other notables: Daphne Koller, Benoit Mandelbrot

Some OSS Projects I Run

  1. Liquid Victor : Media tracking and aggregation [used to assemble this presentation]
  2. Prehensile Pony-Tail : A static site generator built in c#
  3. TestHelperExtensions : A set of extension methods helpful when building unit tests
  4. Conference Scheduler : A conference schedule optimizer
  5. IntentBot - A microservices framework for creating conversational bots on top of Bot Framework
  6. LiquidNun : Library of abstractions and implementations for loosely-coupled applications
  7. Toastmasters Agenda : A c# library and website for generating agenda's for Toastmasters meetings


Everyone Loves a Monolith

Devils Tower - Half Width.jpg

Monoliths Repeat Business Rules

DRY Chalkboard.jpg

SOA Exposed Hidden Problems

SOA Exposed Hidden Problems.png
  Avoiding Dual-Writes - The Problem - Flow - NonAcid - 1280x662.png

Two At-A-Time

Is is NOT possible to reliably make more than one change to system state in a single execution context

Two changes can be made to system state in a single execution context if:

  • The 1st change is idempotent
  • The 2nd change is unreliable

SOA Exposed Hidden Problems

SOA Exposed Hidden Problems.png

Independently Scalable

Process Bottleneck.png

Independently Deployable

Feature Merge.png

Minimize Complexity

Brain Activity 800x688.jpg

What is a Microservice?

  • Exposes common business-rules to multiple application
  • Makes at-most 1 reliable change to system state in an execution context
  • Can scale independent of unrelated concerns
  • Can be owned/deployed/operated by a single small team
  • Can fit inside a developer's head

The Cost of Microservices

Microservices - But at What Cost - HalfPanels.jpg
  • Harder to observe

  • More to manage

  • Eventually consistent


  • Distributed Logging
    • Splunk
    • Raygun
    • Loggly
  • Monitoring
    • DataDog
    • App Insights
    • App Dynamics
  • Actions Required
    • Use a CorrelationId
    • Expose data via abstractions


  • Credential stores
    • Microsoft User Secrets (minimal)
    • Azure KeyVault
    • Hashicorp Vault
  • CI/CD
    • Github Actions
    • Jenkins
    • Azure Devops
  • Instance Management
    • K8S/Rancher
    • AKS
    • Cloud Foundry



The degree to which each observer gets the same view of the state of the system

Consistency Requires Synchrony

space time 800x619.jpg

Synchronization results in Temporal Coupling

Eventual Consistency


The real world doesn't require the consistency we tend to demand of our systems - Kendall Miller 2022-01-14

  Eventual Consistency Post.png

Building Microservices

Contact Machine.gif

The CTP Pattern


Transformer Logic - HTTP vs Kafka

Transformer Logic - HTTP vs Kafka.png

Asynchronous Messaging

Self Healing 600x600.jpg
  • Temporally Decoupled
  • Self-Healing



CTP Pattern Using DAPR

CTP Pattern - DAPR.png

DAPR Bindings

dapr bindings 1165x800.png

Defining Microservices

Storm 1024x683.jpg

Event Storming - A process for modeling a business domain from the perspective of the business experts

Event Storming Output

Final Domain Model - Ask the Sloth.png
  1. Visual Model of the Domain
  2. Ubiquitous Language Document (Dictionary)
  3. Common Understanding Among Team Members

Bus Maintenance System

Bus Maintenance.jpg

WorkOrder Sub-System

  1. Create and capture WorkOrders based on reports from the Transit Reporting system. These WorkOrders should be exposed to the Analytics system for use by our analysts.

  2. Add WorkOrders created due to signals from Bus Telemetry. Minimal data is sent from the telemetry system so the data will need to be augmented with information from the Bus Inventory system.

  3. Add a UI so that users can create/edit/delete WorkOrders.

  4. Add integration with the maintenance system so WorkOrders can be edited and closed from the system that the mechanics use.

  5. Add a Global Search mechanism for WorkOrders that allows searching by nearly any feature of the WorkOrder.

  6. Allow other downstream systems to be notified when changes are made to WorkOrders.

  7. Add an integration to the bus manufacturer's system so they are notified of all failures of their busses.


WorkOrder Sub-System

  • 1 Business Service - Applies Business Rules
  • 4 Ingress Services - Transform to Domain Model
  • 1 Outgress Service - Publish to Vendor

  • 1 DB with at least 2 Tables
  • 2 Kafka Topics (1 public)
  • 2 Kafka Connectors
  • 1 kSQL join

The WorkOrders table is our private (internal) data stream


BSS Seal of Approval.png

Critical C's of Microservices


Defend the execution instance

  • It is impossible to make more than 1 reliable change to system state
  • Never add-on
  • Idempotence is golden
  • Document when any process is less reliable

The Critical C's of Microservices


Embrace the eventual nature of it

  • There is no such thing as "Full Consistency"
  • Our systems RARELY need more than eventual consistency
  • Use Asynchronous messaging whenever possible
  • Don't force your users to wait for an answer
  • Avoid simulating consistency if at all possible

The Critical C's of Microservices


Once a message is defined, all stakeholders have a compatibility expectation

  • Have an internal (non-contract) stream for rapid iteration
  • Isolate internal streams
    • Only one service should write to it
    • Only services in THIS SUBSYSTEM should read from it
  • Use upstream contracts to bring external data to local stores

The Critical C's of Microservices


Failures will occur - Embrace it

  • Play "what-if" games
    • Virtual
    • Using the Simian Army
  • Build self-healing systems

The Critical C's of Microservices


Don't "Roll Your Own"

  • Cross-Cutting Concerns
    • Logging
    • Security
    • Load Balancing
  • Anything not a Core Competency
    • Builds should provide a tactical advantage
    • Event Storming can help define these boundaries

The Critical C's of Microservices


Bring all the operational feeds together

  • Logging
  • Observability
  • Dependency Status

The Critical C's of Microservices


  • Microservices solve reliability and other issues from SOA and earlier architectures
  • A microservice makes at most 1 change to system state
  • Some trade-offs are made when implementing microservices
  • Most microservices use the CTP pattern
  • Idempotency is golden
  • Use Event Storming to help define the subsystems
  • Remember the Critical C's of Microservices


This Slide Deck

Printable Version


Appendix A - Uptime Metrics

  • 3-9's (unimportant)

    • 99.9% uptime
    • 1 outage second in 1000
    • ~ 43 min / month
  • 4-9's (important)

    • 99.99% uptime
    • 1 outage second in 10,000
    • ~ 4.3 min / month
  • 5-9's (critical)

    • 99.999% uptime
    • 1 outage second in 100,000
    • ~ 0.43 min (26 sec) / month

Appendix B - Delivery Guarantees

  • At-Least Once

    • Every messages will be delivered 1+ times
    • The vast majority of message systems make this guarantee
  • At-Most once

    • Every message will be delivered 0-1 times
    • i.e. Logging
  • Exactly once

    • Every message will be delivered 1 and only 1 time
    • Very difficult and expensive
    • Limited usefulness while maintaining this guarantee
    • Combination of idempotent input and a downstream transaction

Appendix C - Two Generals

What The Two Generals Problem tells us about System Reliability

  • There is no such thing as complete synchronization across any 2 parties
    • There is always uncertainty as to whether the most recent message was received
  • Since we will never have complete information, we have to decide when we have enough information to meet our delivery guarantees
    • At Most Once - Message with no confirmation
    • At Least Once - Message with confirmation, resent if not confirmed
    • Exactly Once - At Least Once with atomic storage of Idempotency Keys