Favorite Physicists & Mathematicians

Some OSS Projects I Run

1. Liquid Victor : Media tracking and aggregation [used to assemble this presentation]
2. Prehensile Pony-Tail : A static site generator built in c#
3. TestHelperExtensions : A set of extension methods helpful when building unit tests
4. Conference Scheduler : A conference schedule optimizer
5. IntentBot - A microservices framework for creating conversational bots on top of Bot Framework
6. LiquidNun : Library of abstractions and implementations for loosely-coupled applications
7. Toastmasters Agenda : A c# library and website for generating agenda's for Toastmasters meetings

http://GiveCamp.org

Everyone Loves a Monolith

Monoliths Repeat Business Rules

SOA Exposed Hidden Problems

Two At-A-Time

Is is NOT possible to reliably make more than one change to system state in a single execution context

Two changes can be made to system state in a single execution context if:

• The 1st change is idempotent
• The 2nd change is unreliable

SOA Exposed Hidden Problems

Independently Scalable

Independently Deployable

Minimize Complexity

What is a Microservice?

• Exposes common business-rules to multiple application
• Makes at-most 1 reliable change to system state in an execution context
• Can scale independent of unrelated concerns
• Can be owned/deployed/operated by a single small team
• Can fit inside a developer's head

The Cost of Microservices

Harder to observe More to manage Eventually consistent

Observability

Operations/Management

Consistency

The degree to which each observer gets the same view of the state of the system

Consistency Requires Synchrony

Synchronization results in Temporal Coupling

Eventual Consistency

The real world doesn't require the consistency we tend to demand of our systems - Kendall Miller 2022-01-14

Building Microservices

The CTP Pattern

Transformer Logic - HTTP vs Kafka

Asynchronous Messaging

Temporally Decoupled Self-Healing

DAPR

CTP Pattern Using DAPR

DAPR Bindings

Defining Microservices

Event Storming - A process for modeling a business domain from the perspective of the business experts

Event Storming Output

Visual Model of the Domain Ubiquitous Language Document (Dictionary) Common Understanding Among Team Members

Bus Maintenance System

WorkOrder Sub-System

1. Create and capture WorkOrders based on reports from the Transit Reporting system. These WorkOrders should be exposed to the Analytics system for use by our analysts.

2. Add WorkOrders created due to signals from Bus Telemetry. Minimal data is sent from the telemetry system so the data will need to be augmented with information from the Bus Inventory system.

3. Add a UI so that users can create/edit/delete WorkOrders.

4. Add integration with the maintenance system so WorkOrders can be edited and closed from the system that the mechanics use.

5. Add a Global Search mechanism for WorkOrders that allows searching by nearly any feature of the WorkOrder.

6. Allow other downstream systems to be notified when changes are made to WorkOrders.

7. Add an integration to the bus manufacturer's system so they are notified of all failures of their busses.

WorkOrder Sub-System

• 1 Business Service - Applies Business Rules
• 4 Ingress Services - Transform to Domain Model
• 1 Outgress Service - Publish to Vendor

• 1 DB with at least 2 Tables
• 2 Kafka Topics (1 public)
• 2 Kafka Connectors
• 1 kSQL join

The WorkOrders table is our private (internal) data stream

Recommendations

Critical C's of Microservices

• Context
• Consistency
• Contract
• Chaos
• Competencies
• Coalesce

Context

Defend the execution instance

• It is impossible to make more than 1 reliable change to system state
• Never add-on
• Idempotence is golden
• Document when any process is less reliable

The Critical C's of Microservices

Consistency

Embrace the eventual nature of it

• There is no such thing as "Full Consistency"
• Our systems RARELY need more than eventual consistency
• Use Asynchronous messaging whenever possible
• Don't force your users to wait for an answer
• Avoid simulating consistency if at all possible

The Critical C's of Microservices

Contract

Once a message is defined, all stakeholders have a compatibility expectation

• Have an internal (non-contract) stream for rapid iteration
• Isolate internal streams
  • Only one service should write to it
  • Only services in THIS SUBSYSTEM should read from it
• Use upstream contracts to bring external data to local stores

The Critical C's of Microservices

Chaos

Failures will occur - Embrace it

• Play "what-if" games
  • Virtual
  • Using the Simian Army
• Build self-healing systems

The Critical C's of Microservices

Competencies

Don't "Roll Your Own"

• Cross-Cutting Concerns
  • Logging
  • Security
  • Load Balancing
• Anything not a Core Competency
  • Builds should provide a tactical advantage
  • Event Storming can help define these boundaries

The Critical C's of Microservices

Coalesce

Bring all the operational feeds together

• Logging
• Observability
• Dependency Status

The Critical C's of Microservices

Summary

• Microservices solve reliability and other issues from SOA and earlier architectures
• A microservice makes at most 1 change to system state
• Some trade-offs are made when implementing microservices
• Most microservices use the CTP pattern
• Idempotency is golden
• Use Event Storming to help define the subsystems
• Remember the Critical C's of Microservices

Resources

• Code Samples
• Idempotence
• The Transactional Outbox Pattern
• Cosmos DB Change Feed Processor
• The Two Generals Problem
• Data Streaming
  • Redis Streams
  • Azure Service Bus
  • RabbitMQ
  • Kafka

This Slide Deck

http://IntroToMicroservices.azurewebsites.net

Printable Version

Appendix A - Uptime Metrics

• 3-9's (unimportant)

  • 99.9% uptime
  • 1 outage second in 1000
  • ~ 43 min / month

• 4-9's (important)

  • 99.99% uptime
  • 1 outage second in 10,000
  • ~ 4.3 min / month

• 5-9's (critical)

  • 99.999% uptime
  • 1 outage second in 100,000
  • ~ 0.43 min (26 sec) / month

Appendix B - Delivery Guarantees

• At-Least Once

  • Every messages will be delivered 1+ times
  • The vast majority of message systems make this guarantee

• At-Most once

  • Every message will be delivered 0-1 times
  • i.e. Logging

• Exactly once

  • Every message will be delivered 1 and only 1 time
  • Very difficult and expensive
  • Limited usefulness while maintaining this guarantee
  • Combination of idempotent input and a downstream transaction

Appendix C - Two Generals

What The Two Generals Problem tells us about System Reliability

• There is no such thing as complete synchronization across any 2 parties
  • There is always uncertainty as to whether the most recent message was received
• Since we will never have complete information, we have to decide when we have enough information to meet our delivery guarantees
  • At Most Once - Message with no confirmation
  • At Least Once - Message with confirmation, resent if not confirmed
  • Exactly Once - At Least Once with atomic storage of Idempotency Keys

Contents

• Introduction
• How did we get here?
• What is a Microservice
• Microservice Trade-Offs
• Building Microservices
• Defining Microservices
• Bus Maintenance System Sample
• Recommendations - the Critical C's of Microservices
• Summary
• Resources
• Appendices