MemoryBlobStore limits the delegate store to a single thread

[brettniven]

The MemoryBlobStore limits the delegate store to a single thread, which can degrade performance. It effectively forces all 'put', 'get' and other operations to be bound to a single queue.

I've written a sample test and a one-liner change to highlight the problem. This is here: brettniven@b7f08e2. By increasing the executor threads, it obviously has much better performance.

I can assist on a fix - I'm just not sure on the best approach as I can't figure out why it would have the '1 thread' limit in the first place. Some links to explain what is happening:

• MemoryBlobStore constructs executorService with a pool limit of 1:

  geowebcache/geowebcache/core/src/main/java/org/geowebcache/storage/blobstore/memory/MemoryBlobStore.java

  Line 99 in fb01245

  this.executorService = Executors.newFixedThreadPool(1);

• All put/get/other operations invoke executeBlobStoreTask, which waits, here:

  geowebcache/geowebcache/core/src/main/java/org/geowebcache/storage/blobstore/memory/MemoryBlobStore.java

  Line 611 in fb01245

  executed = future.get();

• The executorService is documented in comments here:

  geowebcache/geowebcache/core/src/main/java/org/geowebcache/storage/blobstore/memory/MemoryBlobStore.java

  Line 52 in fb01245

  * relation with the underlying wrapped {@link BlobStore}. In fact, each operation on the wrapped

... but I can't work out a reason for it needing to be limited to 1 thread. Each delegate store should have it's own locking code, where applicable.

For reference, our setup is:

• GWC embedded in GeoServer, 1.24.x, using Kartoza docker image
• We're hosting in Azure, using an Azure File Share (which unfortunately for us, is 'slow' for a few potential reasons, which exacerbates this issue - we're in the process of trialling the Azure Blob Store plugin)
• We're using a MemoryBlobStore as a 2-layer cache, fronting a FileBlobStore. i.e. innerCachingEnabled=true.
• We investigated performance and found that we get substantially better performance when we turn the MemoryBlobStore off, which is a very simple workaround for this issue. However... we believe we've also found a thread deadlock issue in the FileBlobStore code (which I'll raise in another issue when time permits)

[aaime]

@brettniven I don't know either (some research would be needed) but:

• I also have the impression the code is over-using locks and concurrency limitations
• The class is 10 years old
• The original author was well-meaning, but also inexperienced. Maybe he wanted to avoid potential double-computation of the same tile? But that's the job of other parts of GWC.

Generally speaking, AFAIK a BlobStore implementation should be thread-safe, and a wrapper should not need to care for synchronizing/serializing access to it. @smithkm can you provide any more information/insight?