Non-linearizable behavior in `cancel` + `awaitClose` inside of `produce`

[globsterg]

Describe the bug

The cancel operation on the channel returned by the produce function, when used together with awaitClose, leads to a data race:

• If cancel only manages to cancel the channel, awaitClose returns normally.
• If cancel also cancels the job of the produce coroutine, awaitClose throws a CancellationException.

In most cases, the code after awaitClose inside produce will not execute if the channel gets cancelled, and it's possible that someone could start relying on this behavior, even though it is not guaranteed.

It looks like cancelling the coroutine first and cancelling the channel later inside cancel may fix this particular bug, but I don't understand if the current order of operations, too, has its upsides.

I do not know if this actually affects anyone, I discovered this analytically while working on #4148

Provide a Reproducer

In the current develop branch, add this to ProduceTest.kt:

@Test
fun produceAwaitCloseStressTest() = runTest {
    repeat(100) {
        coroutineScope {
            val c = produce<Int>(Dispatchers.Default) {
                try {
                    awaitClose()
                    println("Normal exit")
                } catch (e: Exception) {
                    println("Exception $e")
                    throw e
                }
            }
            launch(Dispatchers.Default) {
                c.cancel()
            }
        }
    }
}

I get both exceptions and (much more rarely) normal exits reported when I run this.

[fzhinkin]

@globsterg, thanks for reporting the issue!

In most cases, the code after awaitClose inside produce will not execute if the channel gets cancelled, and it's possible that someone could start relying on this behavior, even though it is not guaranteed.

Indeed, docs state that no code after canceled awaitClose will be executed:

Therefore, in case of cancellation, no code after the call to this function will be executed.