headers_sent - behavior changed from PHPUnit 9 to 10

[DanielRuf]

Q	A
PHPUnit version	10.x
PHP version	8.1.13
Installation Method	PHAR

Summary

Starting with 10.0, tests with headers_sent() have a different behavior now.

Current behavior

The headers_sent() result differs from PHPUnit 9 to 10.

How to reproduce

<?php declare(strict_types=1);
use PHPUnit\Framework\TestCase;

final class BasicTest extends TestCase
{
	public function testEqual(): void
	{
		$string = 'a';

		if (!headers_sent()) {
			$string = 'b';
		}

		$this->assertSame($string, 'a');
	}
}

Expected behavior

The check for headers_sent() does not change from PHPUnit 9 to 10.

For reference, the results when testing with the original phar of PHPUnit 9.6.22 and 10.5.44:

php.exe .\phpunit-10.5.44.phar ./BasicTest.php
PHPUnit 10.5.44 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.1.13

F                                                                   1 / 1 (100%)

Time: 00:00.039, Memory: 26.00 MB

There was 1 failure:

1) BasicTest::testEqual
Failed asserting that two strings are identical.
--- Expected
+++ Actual
@@ @@
-'b'
+'a'

...\BasicTest.php:14

FAILURES!
Tests: 1, Assertions: 1, Failures: 1.

php.exe .\phpunit-9.6.22.phar ./BasicTest.php
PHPUnit 9.6.22 by Sebastian Bergmann and contributors.

.                                                                   1 / 1 (100%)

Time: 00:00.006, Memory: 26.00 MB

OK (1 test, 1 assertion)

I already tested and this affects every PHPUnit 10 version, including 10.0.0.

I checked the changelog and issues, so far I found no changes that could lead to this.

[DanielRuf]

By using the first parameter of headers_sent() I can see, that in PHPUnit 9 the headers were sent in phar://.../phpunit-9.6.22.phar/phpunit/Util/Printer.php, which is not the case in PHPUnit 10.

[DanielRuf]

What is the recommended solution to prevent that the tests fail but the application code still works? Maybe adding some check if the current SAPI is the CLI (similar to the following snippet)?

if (php_sapi_name() !== 'cli' && !headers_sent($file)) {
	$string = $file;
}

[staabm]

what are you trying to achieve by checking for headers_sent? it feels wrong to check for such a global side-effect and rely on it

[DanielRuf]

what are you trying to achieve by checking for headers_sent? it feels wrong to check for such a global side-effect and rely on it

The web application code is ~20 years old and depending on the result it adds a header or not.
To me that does make sense. Or how would you check and prevent for "Headers already sent"?

[staabm]

based on the information given in this PR I can't say more then it feels wrong that a test relies on headers_sent, which is a assumption which can break pretty easily and its pretty hard to find the root cause why (fragile test).

I agree though, that PHPunit itself should "not send headers" (in case its technically possible) - but thats need to be investigated.

[sebastianbergmann]

it feels wrong that a test relies on headers_sent

It's common and not wrong (maybe not clean) in end-to-end tests. It used to work and I do not know why it no longer does. I would like to investigate that and ideally restore the previous behaviour.

[DanielRuf]

based on the information given in this PR I can't say more then it feels wrong that a test relies on headers_sent,

Not the test itself, the application code. My test in this issue is just a simple reproducible example. The real test does not directly use headers_sent but the application code used by it. At least in my case (and there our test asserts the result, which now differs due to this change).

[mvorisek]

headers_sent() is pointless in CLI and I would expect the default value on php startup unchanged.

Also note headers_sent() cannot be changed back once headers are sent.

[MrSuddenJoy]

headers_sent() is worthless as its outdated. There are newer, more modern ways of benchmarking webapps.