[FEATURE]: Better progress for compressed archives

[bryceschober]

Is your feature request related to a problem? Please describe.

The progress output when writing compressed images files quickly progresses to 99% and then pauses there while the rest of the disk is being written.

Describe the solution you would like to see implemented

I've seen this issue comment noting that this is expected behavior for compressed archives. On the other hand, I get the correct uncompressed size from the output of both gzip -l my_os.img.gz and zstd -vl my_os.img.zstd, and it seems that rpi imager could use the equivalent functionality...

Yeah, I know, I'm a programmer too. It's easy to say in one sentence of english. Implementing is another thing entirely. 😄

Describe alternatives you've considered

Live with uninformative progress display, or decompress it first. 😒

Additional context

None

Version

1.9.0

[lurch]

I get the correct uncompressed size from the output of both gzip -l my_os.img.gz

Unfortunately gzip only stores the uncompressed filesize modulo 4GB; so if you have a 5GB disk-image compressed with gzip, gzip -l would report it as 1GB 🙁

[bryceschober]

Unfortunately gzip only stores the uncompressed filesize modulo 4GB; so if you have a 5GB disk-image compressed with gzip, gzip -l would report it as 1GB 🙁

Well size % 4GB is still perhaps more informative than nothing. I'd also be perfectly happy if it were only working for compression options that contain correct headers size, and that being documented.

I feel like the bigger question is whether the libarchive interfaces expose that functionality for you already, and it could relatively easily implemented or not.

[lurch]

Well size % 4GB is still perhaps more informative than nothing.

Not necessarily - you might have a 5GB mostly-empty disk image which compresses down to 750MB, or you might have a 1GB not-very-compressible disk image which compresses down to 750MB. In both cases gzip would tell you that the uncompressed file is only 1GB. And a user would be very confused if they tried to write that file to a 4GB microSD card, and it ran out of space.

[bryceschober]

And a user would be very confused if they tried to write that file to a 4GB microSD card, and it ran out of space.

Okay, I don't really mean to be argumentative, but the current state is such that the progress is based on the compressed size? We're only talking about the progress, right? Not some kind of validation for the user.

Out of curiosity, I downloaded the 2024-11-19-raspios-bookworm-armhf-full.img.xz and imaged it to my SD card, and its progress does seem to reflect the full uncompressed write time, at least the time stuck at 99% was much less than the difference in compressed vs. uncompressed size. I get why that behavior isn't available for .gz files, but now I'm kinda confused why the same isn't already available for .zstd files, which do list their correct uncompressed size.

[lurch]

I'm not very familiar with RPi Imager's internals, so I think I'll have to leave @tdewey-rpi to answer any questions here 😉