Redis part 2 - Application cache benchmark: NVMe SSDs and memcached

Why these Redis benchmarks?

In our daily work with online shops, we have seen time and again that the use of Redis has led to an increase in performance. The throughputs of write, read and delete operations were improved by Redis, which was proven by earlier benchmarks by Redis developer Colin Mollenhour. But technology continues to evolve and so does the hardware we use: we use NVMe SSDs as data storage on our clusters. They achieve much higher data throughputs than SATA SSDs or HDDs and therefore lead to better performance in e-commerce. The data storage is used by the shop systems Magento and Shopware as a cache backend by default. Redis and memcached, on the other hand, store the data structures in the main memory. With this benchmark, we wanted to find out which form of data storage for application caches ultimately leads to better performance for online shops: Redis (main memory), memcached or NVMe SSDs (file system).

For this, we measure the throughput of erase, write and read operations per second and compare them.

In the second benchmark, which we will present in the next blog post "Redis part 3", we focused on the sessions. We wanted to find out which session storage is more performant when reading, writing and changing data: Redis, MySQL or NVMe SSDs.

Test environment

The tests were carried out on one of our clusters with our current hardware (at the time of the test), such as Intel Xeon Gold 6254. For the test environment, we chose 4 CPU cores to represent the computing power of one of our average clusters. We set the RAM height to the very high value of 32 GB so that all cache files could be stored in the Linux file system cache. In most cases, this is possible with much less RAM.

Benchmark: Application caches under Magento 1

Since Magento 1 has a complex cache system, we decided to run the benchmark under Magento 1 as an example. This allows us to demonstrate the effects of cache backends on complex data structures. The benchmark for application caches was carried out under the (at the time of the test) current version Magento 1.9.4.1.

Shopware and Magento online shops shop the data in the data memory (NVMe SSDs in our case) by default. Redis, on the other hand, stores the data in the main memory. In order for Redis to be supported by Magento 1, certain plugins must be installed. Magento 2 and Shopware already support Redis natively.

Colin Mollenhour's benchmarks from 2012 served as the starting point for our test. We replicated this test on our hardware and the current Magento 1 shop system and tested various backends.

Backends for the benchmark

Zend_Cache_Backend_File

The backend Zend_Cache_Backend_File is used with Magento 1 if no other special backend is used via the lokal.xml or a module. A special configuration is not necessary here.

For our tests, we ran the benchmark on the fast NVMe SSDs we used.

Cm_Cache_Backend_Redis

This is a module by Collin Mollenhour, which allows the Magento 1 cache to be stored in Redis. For Magento 2, an implementation on the same code basis is already integrated in the shop system. Shopware also allows the cache to be stored in Redis without further modules.

memcached (Zend_Cache_Backend_TwoLevels) + Zend_Cache_Backend_File

Magento 1 allows the cache to be split into two parts according to fast and slow backend. This allows a part of the cache to be placed in a fast backend that does not support tags. The remaining part of the cache, which needs the tags, is placed in another, slower backend. Without additional modules, it is possible to use the key-value database memcached as a fast backend for this case. For this test, we used memcached as the fast backend and the file system as the slow backend.

memcached (Zend_Cache_Backend_TwoLevels) + Cm_Cache_Backend_Redis

As a fourth variant, we tested how memcached and Redis behave in combination. We have therefore activated the TwoLevels backend again, but entered Redis as the slow cache for this case.

After configuring the cache benchmark, elements were then created for the benchmark:

php -f shell/cache-benchmark.php --keys 200000

The actual benchmark was then carried out and evaluated.

bash var/cachebench/default/run.sh

We use a cache size of 200,000 keys in the test, which in our experience corresponds to an average Magento shop.

Results

A write operation stores a single item in the cache, a read operation reads it. A delete operation removes all elements from the cache that are provided with a specific cacheIn contrast to the other backends, the support of complex data structures in Redis makes it possible to find elements via tags. For this purpose, the cache backend creates a set for each tag in which all associated keys are saved. In addition, there is a set that contains all tags. In this way, it is extremely efficient to remove all keys belonging to a tag. This method is not available for file system-based backends or memcached, so the number of deletion operations is much lower for these backends (see diagram 1). Due to the inefficient structure of the standard Magento application cache (default setting), the throughput of delete operations is extremely low.

In contrast to the other backends, the support of complex data structures in Redis makes it possible to find elements via tags. For this purpose, the cache backend creates a set for each tag in which all associated keys are saved. In addition, there is a set that contains all tags. In this way, it is extremely efficient to remove all keys belonging to a tag. This method is not available for file system-based backends or memcached, so the number of deletion operations is much lower for these backends (see diagram 1). Due to the inefficient structure of the standard Magento application cache (default setting), the throughput of delete operations is extremely low.

This has an effect on the speed. The Cm_Cache_BackendRedis module achieves a much higher speed than the other backends, as delete operations can be carried out quickly via any tags due to the structured data. With file system-based caches, on the other hand, it is much more cumbersome to remove all elements of the cache that have a specific tag, as the corresponding files cannot be found directly via a tag.

Diagram 2 shows the number of read operations. It is highest for file system-based cache backends. This is due to the very efficient file system management of Linux, which automatically keeps files in the main memory if free memory is available. In our test environment, we have provided a very high working memory of 32 GB so that there is always enough memory available for our test. This allows files to be read extremely efficiently. In a real situation with limited RAM, this is not always the case. In such a case, Redis offers the advantage of ensuring that the application cache is in RAM at all times.

Diagram 3 shows the number of write operations per second. The differences between the individual backends are smallest in this throughput test. However, Redis offers the advantage of storing data quickly in RAM due to its simple data structures.

Conclusion: Is Redis the best option as an application cache?

At first glance, it is obvious that Redis is more inperformant in the area of read operations due to the lower throughput. However, Redis has a higher throughput in the area of write operations and a significantly higher throughput in the area of delete operations.

The great advantage of Redis becomes apparent when comparing the number of operations per second among each other. The relevance of the individual operation types must be taken into account. The relevance of delete operations is much higher than that of the other operations. This is because ultimately, every time a product is purchased in the online shop, elements from the cache lose their validity and must therefore be deleted. Subsequently, the previously deleted cache elements have to be rewritten, i.e. write operations. Here, too, Redis shows a higher performance. The throughput of write operations is higher with the Redis backend than with the other backends.

NVMe SSDs have the highest number of read operations per second as an application cache. For the erase and write operations, NVMe SSDs offer a lower throughput. However, these operations have more impact on a shop with a high conversion than the read operations. In this respect, the erase and write operations have a higher relevance in e-commerce. For online shop operators, the values of the delete and write operations are therefore particularly important, in which Redis offers clear advantages according to our measurement results. For this reason, we recommend the implementation of Redis for online shops. Especially for medium and large online shops.

For the key-value database memcached as a backend, the results in both two-level deployment scenarios show that memcached does not have a better overall result compared to the other backends. Only memcached offers a higher throughput in the read operations. Since memcached has a higher level of complexity in setup and operation as an application cache, we cannot make a recommendation for memcached after our tests, since there is no great advantage for online shops with this setup effort.

For Showpare, Magento 1 and Magento 2 online shops, we recommend the use of Redis for efficient storage of application caches in the main memory after our test. For smaller shops, NVMe SSDs already lead to sufficiently good performance. The combination of fast SSDs for loading PHP files and static content and Redis as an application cache represents the best environment for a high-performance operation of online shops.

Tabular recommendation for the different shop systems and shop sizes

More posts in this blog series

• Redis Part 1 - Performance optimization for Shopware and Magento
• Redis Part 3 - Sessions Benchmark - NVMe SSDs, Redis and MySQL
• Redis Part 4 - How is Redis integrated into Shopware?
• Redis Part 5 - How to integrate Redis into Magento?

Titelbild: raspirator/ iStock.com

Last updated on 09.06.2021 | JH