Hiding instances on MySQL InnoDB Cluster/ReplicaSet

MySQL Router is a core component of InnoDB Cluster/ReplicaSet that enables the automatic routing of client traffic to the right instances of your database architecture. Router can automatically adapt to topology changes, however, there are some scenarios on which one would certainly benefit from having the possibility to tag an instance with a certain attribute that indicates a specific behavior change to Router’s default behavior.

In the 8.0.21 release, we have introduced a tagging framework to the AdminAPI which ultimately allowed us to introduce a very demanded feature:

  • Hide Cluster/ReplicaSet instances from client traffic

Why would one want to hide instances?

The ability to mark an instance as unavailable to receive traffic, temporarily, is very important as it enables:

  • Performing rolling upgrades on servers without disrupting incoming traffic
  • Performing maintenance operations or configuration changes on servers without having to stop MySQL
  • Exclude a server from the client workload which is lagging behind
  • Excluding database servers located in a different zone/region for Disaster Recovery purposes
  • Excluding a SECONDARY server from the read workload while backups are taken or reports are generated to avoid affecting other queries.
Upgrades/Maintenance

Performing a rolling upgrade of MySQL requires taking the instance offline. However, one has to worry about the possible connection failures to that instance. Excluding the instance from the routing destination candidates list allows DBAs to safely take the server offline and ensure no application/router traffic is sent to it.

Depending on the maintenance task, it might not even be necessary to take the server offline. Up to now, that implied stopping Group Replication or MySQL itself to avoid disrupting incoming traffic that could be affected in terms of performance. By excluding the instance from the routing candidates, this can be completely avoided.

Replication Lag

Considering that InnoDB ReplicaSet is based on asynchronous replication, replication lag is a common factor to be taken into consideration. If an instance is lagging behind, the DBA may want to hide it from applications until it catches-up, and replication lag is reduced. The same applies to InnoDB Cluster, though Group Replication is virtually synchronous and replication lag is measured differently.

 Disaster Recovery and Backups

More complex architectures include servers that are used for backup purposes or for Disaster Recovery. But those servers should not get any workload or should be temporarily excluded from client traffic meanwhile a backup is being performed.

That is now possible, by excluding those servers from the routing candidates list.

NOTE: Secondary 2 is being used to take a backup, so the DBA excludes it from incoming client connections

Data reports

Generating data reports on a specific instance can be very demanding and impact the instance’s performance. For that reason, the recommended approach is to exclude that server from the routing candidates list.

OK, but how?

MySQL Router is constantly pooling the Metadata for configuration changes, so using the tagging framework is undoubtedly the right approach to allow such customizations.

With the help of the AdminAPI and its ease-of-use, users can now set tags to instances with a simple call to .setInstanceOption().

For the purpose of hiding instances from client traffic, we have introduced two “built-in” tags that enable you to immediately change the Router’s behavior:

  • _hidden
  • _disconnect_existing_sessions_when_hidden

By enabling the _hidden tag on a specific instance, you instruct MySQL Router to exclude the instance from the list of possible destinations for client applications. The effect is virtually immediate considering that MySQL Router is constantly polling the Metadata.

By enabling the _disconnect_existing_sessions_when_hidden tag on a specific instance, you instruct Router to disconnect existing connections from instances that are marked to be hidden.

NOTE: By default, _disconnect_existing_sessions_when_hidden is enabled.

Conclusion

Hiding instances is a feature unquestionably important for any production deployment of MySQL InnoDB Cluster and ReplicaSet. It allows performing many laborious tasks without disrupting the client’s traffic and on top of that, enables more complex setups.

Please upgrade your deployments to 8.0.21 and enjoy the new exciting features!

Thank you for using MySQL!

About Miguel Araújo

Miguel Araújo is a Principal Software Engineer on the MySQL Team, at Oracle. He's the Tech Lead of the AdminAPI, core component of MySQL InnoDB Cluster, at the MySQL Shell team. In the past, he has worked on different projects and teams, mostly related to Middleware and High-Availability. He has a Computer Science Engineering degree and Master's degree, from the University of Minho, Portugal, where he was also a researcher. His backgrounds are on distributed systems, scalability, database replication and high-availability. He is based in Portugal.

3 thoughts on “Hiding instances on MySQL InnoDB Cluster/ReplicaSet

Leave a Reply