August 19th, 2016
(written by lawrence krubner, however indented passages are often quotes). You can contact lawrence at: email@example.com
Three years ago, I was intrigued to read that Twitter had implemented dynamic network topologies. Something similar is becoming more common, thanks to the use of Nginx/HAProxy. One can use this for many things, such as connecting to a pool of MySQL servers.
However, there are some downsides worth remembering:
You have to be careful to tune your xinetd script to increase the cps and per_source limits accordingly to the load that your pool of servers generate with this proxy forwarding.
Assuming that all servers that need to connect to MySQL have this local tcp 3306 proxy setup, each time you increase your pool of servers the load of incoming check requests increase as well.
Xinetd is quite nasty and by default will stop responding to requests for like 10 seconds after some defaults. This will effectively “block” your whole database connection flows, especially when you restart services at batch during a rolling update, and under some other circumstances…