Add support for MemoryMapped Files.

[Cyberavater]

Very self-explanatory, typical write API can be very CPU extensive when writing a small amount in each iteration.
The solution is MemoryMapped Files. most platforms support it.

For JVM, I recommend against using the JDK-provided MappedByteBuffer and implementing it from scratch with JNA/JNR.
MappedByteBuffer cons:

1. You cannot unmap at will while it's very possible to do so if you call the native API for that. [This is useful if you want to have a dynamic-sized mapped file, ie take a reasonable-sized chunk, and if it's more than needed you can just unmap and resize internally, JDK 8 implementation doesn't provide any solution for that, but Panama solves it, which is not usable on android]
2. Bulk write on Android SDK is missing, so for that you'll have to write JNA either way.

[amal]

Universal KMP MemoryMapped files API can be huge!

But I'd say JDK MappedByteBuffer can be a nice option to start with, but JNA/JNR may be even better for sure. Just some users will find it easier to just stick to JDK one.

JDK MappedByteBuffer can be unmapped with some reflection magic on most of the platforms (Android, various JVMs). I can help with that. But it's not 100% reliable because of this hacky way.

[fzhinkin]

The main obstacle on the way of supporting mmapped files is how to fit them into existing library design. If you have any particular use cases in mind, or a project that already relies on memory mapped IO, sharing it would help a lot with designing the API.

[amal]

@fzhinkin, I’ve been using memory-mapped files heavily in my reading application for years. It’s the fastest IO for my cases.

It is a closed source, but for my cases it’s no different than regular random access to files.

I have a universal interface for different implementations. And I’ve started to open source the IO layer, so there are some examples. I’m just about to add implementations with kotlinx-io :)

1. Here is a NIO ByteBuffer-based RAD/RandomAccessData implementation, which also covers the mmaping.
   fluxo-kt/fluxo-io/../rad/ByteBufferRadAccessor.kt

2. Tests for it, including regular buffers and mmaped files:
   fluxo-kt/fluxo-io/../rad/RandomAccessDataByteBufferTest.kt

3. And here is a universal cleanup (unmapping) for JVM byte buffers with support for different JDK and Android versions:
   fluxo-kt/fluxo-io/../nio/BufferUtil.kt#L27

[Cyberavater]

JDK MappedByteBuffer can be unmapped with some reflection magic on most of the platforms (Android ....

I highly doubt that it'll work on Java 9 Android versions ie from SDK 28. The reason what I understand is those tricks from java 9 require JVM start arg which can NOT be done on Android, so I didn't even try it and suggested JNA solution.

But if you tested it and it works as expected then you're right and my understanding is wrong.

[amal]

@Cyberavater, for Android API level 27+ there is an actual public API for unmapping.
android.os.SharedMemory { public static void unmap (ByteBuffer buffer) }

It works well. Actually, it worked quite nicely for many years already.
Here is how I use it:
fluxo-kt/fluxo-io/../nio/BufferUtil.kt#L188

The problem is, you need to collect all these hacks and SDK-specific APIs in one place to work. But I’ve collected it from different places already :)

Before that, there was a grey list Android API method.
java.nio.NioUtils { public static void freeDirectBuffer(ByteBuffer buffer) }

And for even older Androids (4.1+) there was a trick with MappedByteBufferAdapter.

[Cyberavater]

@amal You are right, thanks! You've done wonderful work. I hope we get a stable release soon, all the best!

[KitsuneAlex]

At least for Kotlin/Native, i have a good implementation that's based on kotlinx.io as a possible starting point. You could easily adapt this to Kotlin/JVM too.