Microservices: refactoring your teams
The speakers at the Microservice Practitioners Summit shared a lot of hard-won expertise about how to scale up and speed up microservices’ agility and performance, with an emphasis on observability.
Matt Klein explained how routing all networking for Lyft’s microservices through Envoy made code pathways observable, increasing trust in microservice calls. Varun Talwar shared the performance advantages of gRPC and how it enables deadline-based scheduling to keep Google’s response time under 400ms. Josh Holtzman’s success in shifting to microservices at xoom.com was enabled by deep monitoring to measurably improve throughput.
What I didn’t expect was so much thoughtful discussion of the mutual impacts of team organization and code organization. Christian Posta warned of a mismatch of culture between Enterprise IT and microservices, and quoted Adrian Cockcroft:
People try to copy Netflix, but they can only copy what they see. They copy the results, not the process.
This was reinforced by Rafi Schloming’s narrative of datawire’s journey from being technology focused to realizing that ‘Microservices are about enabling a developmental methodology for systems’. What is needed is Service Oriented Development, not Service Oriented Architecture, and that to achieve this you need small teams with Service Level Objectives.
Nic Benders took this a stage further- he invoked Conway’s law:
Organizations which design systems… are constrained to produce designs which are copies of those organizations
His engineering organization at New Relic mapped out the dependencies between the different microservices and teams there:
They then simplified and clustered the service and team diagram into a smaller number of teams with fewer cross-team dependencies, and then allowed the engineers to pick which teams they worked on. Each team was given more autonomy on how to work together, and control of their Minimal Marketable Features. Nic said:
You hired smart engineers — trust them. We didn’t do this with MBAs and VPs but with the teams themselves.
Susan Fowler took up the Conway’s Law theme, but with a twist — based on her experience at Uber, she described a ‘reverse Conway’s Law’ where the 1000+ microservices influenced the team dynamics. She warned that this lead to organization siloing and sprawl — microservices developers become like services and very siloed. Unless you standardize operational models and communication, they won’t be able to move teams.
Her prescription was more standardization of the framework that the microservice developers worked within.
The way to approach this is to pick a goal, such as availability of services. This implies stability, reliability and scalability is needed, which leads to standardizing on systems that provide the kind of observability into service flows and interactions that the earlier speakers have described.
As Matt Klein put it:
Robust observability and easy debugging are the most important thing — without that devs don’t trust the system.
Varun Talwar concurred:
You want observability and stats — you need a common nomenclature to make sense of large call graphs. Tracing a request through services is key — you often have 1 query out f 10,000 that is slow and you want to trace it through the whole call chain.
Rafi Schloming summed it up:
Start with the people and think how to divide up the work first, let that lead to the technical perspective.
