Random access to files

[LewsTherinTelescope]

Multiplatform Sources and Sinks are already great features, but it would also be nice to have random-access capabilities like Okio's FileHandle class. The BufferView option discussed in #225 would help in some cases, but in other cases it's impractical to load the entire file into memory to get at a small part of it—reading a few individual files from an archive, for example. As far as I know the only way to do this right now is to create a new Source and skip to the specified offset, which isn't particularly convenient. (In fact, if I'm reading the code correctly, the skip function does read everything up until that point into a buffer temporarily? Which would make it a no-go for those usecases as well.)

(I've tried to search both the repo and Slack to make sure there wasn't existing discussion about this, but if I've missed something please point me to it!)

[ptitjes]

@fzhinkin I also am in need of this feature, as I am also using Okio's FileHandles in my projects. I think Okio's current design on this is rather genuine. Would you accept a PR that replicates their design?

[fzhinkin]

@ptitjes there is a plan to add proper file support to kotlinx-io (replacing or extending existing API that was added mainly to plug a few specific use cases), but the corresponding API should be designed first. The latter means that Okio's FS API will be considered but will be not necessarily taken as is.
At this point, I would abstain from taking FileHandle from Okio until it is clear what future kotlinx-io's file support will look like.

[bjhham]

I was also looking for random access files, so I ended up writing a pretty simple implementation to get me by for the JVM. Maybe you could consider it in the design.

Example usage:

fs.randomAccessStorage(file).use { storage ->
    Buffer().use { buffer ->
        while (storage.readAtMostTo(buffer, blockSize) == blockSize) {
            storageAdapter.read(buffer)?.let { row ->
                if (query(scope + row)) {
                    // roll back to start
                    storage.position -= blockSize
                    // write the new item
                    val newRow = fromRow(row.update(mod.changes))
                    storageAdapter.write(newRow, buffer)
                    storage.write(buffer, blockSize)
                    index[newRow] = storage.position
                }
            }
        }
    }
}

Code:
See https://pl.kotl.in/TI-mCJ5mc

[amal]

@bjhham You can take a look at fluxo-io. It provides different RandomAccessData implementations (nine options for JVM, and one cross-platform). I plan to add kotlinx-io based cross-platform implementation soon.