Atomics (or GCC intrinsics) were first introduced in InnoDB (5.0) by a patch from Mark Callaghan’s team at Google for mutexes and rw-locks. InnoDB code then was written in C. When the code was ported to C++ , part of the 5.6 release, there was no C++ standard for atomics. Over time this led to a hodge lodge of code tweaked for different platforms, hardware, operating systems and compilers.
There were many things wrong with what we ended up with. This old code didn’t handle all types, the supported types were hardcoded for a few platforms and compilers only, with fallback to mutexes for all other access. Memory ordering rules were difficult to reason about because some of the GCC intrinsics were more strict than required and this made the rules unclear, we usually tried to take the more conservative approach. This would sometimes be at the cost of some performance. What is more important we did not have means to force developers to always access a given variable through atomic operation – the decision to use atomic or non-atomic access was taken independently for each access. Therefore there were some variables on which some operations were performed atomically and some not, causing to be a UndefinedBehaviour at best.
This change to move all atomic operations to the C++11 standard has increased portability – we use well tested code from C++ standard and don’t need to reinvent the wheel anymore.
There are more advantages of this change, for example we didn’t have atomic operations for
bool, and in
rw_lock_t we used
ulint for field
waiters, for which only values 0 and 1 were used. Now it’s defined as
std::atomic<bool>, which reduced the size of the
rw_lock_t structure. This has the nice side effect off reducing the size of the meta data in the buffer pool so that we can use the buffer pool more efficiently.
As mentioned earlier, there were some challenges with variables accessed with both atomic and non-atomic operations. For example there was atomic usage of a MYSQL_SYSVAR
srv_fatal_semaphore_wait_treshold which couldn’t just be changed to std::atomic. Therefore we have split the variable into two: a base regular SYSVAR variable and an atomic variable:
srv_fatal_semaphore_wait_extend. So now our base value works as SYSVAR, while the temporatary changes done by the code are done atomically to the additional variable. While on the subject, we analyzed the code around this variable. We did some refactoring there, and it turned out that
srv_fatal_semaphore_wait_extend simplified our code.
We also did some changes in monitor counters which you can observe via
INFORMATION_SCHEMA.INNODB_METRICS. We had
MONITOR_ATOMIC_DEC which used atomic increment/decrement. Now we changed
mon_value in structure
std::atomic<mon_type_t>. But we had also
MONITOR_DEC_VALUE which also incremented/decremented
mon_value non-atomically – we wanted it to be faster, and exact value wasn’t so important. So in this case we had to use .load() and .store() with
std::memory_order_relaxed to leave the old behavior. Therefore we have:
const auto new_value =
MONITOR_VALUE(monitor).load(std::memory_order_relaxed) - value;
This differs from
MONITOR_VALUE(monitor).fetch_sub(value, std::memory_order_relaxed), because the latter will be executed atomically.
Using the opportunity while working on this area we also did some refactoring here, replacing some of #defines with inline functions.
This work will help us create more robust software across different platforms and not maintain additional code that is just available in the C++ standard.
Thank you for using MySQL!