How to change complex network systems once you’ve made important decisions?

(written by lawrence krubner, however indented passages are often quotes). You can contact lawrence at:


Lets use service discovery as an example. We’ve used service discovery along with health checks and self diagnostics to be able to do some fairly interesting things during outages and, more importantly, slowdowns. These are hard problems, and we’ve solved them rather crudely in places, but these solutions still give us a level of proven, necessary sophistication for system stability.

Unfortunately, our approach is slightly incompatible with Kubernetes’ centralized one. Usually, when this happens, your needs are just too sophisticated, but in this case Kubernetes’ approach is already explicitly complex in order to try to deal at the proper level of sophistication. It’s a thoughtful and mature approach, but its structure is just inverted from ours. It’s complex in incompatible ways, the worst of both worlds.