[Discussion] Desired texture/sampler state

[lexaknyazev]

Since KTX2 already has swizzling metadata, it's fair to explore what other runtime properties may need to be stored in the same way (using optional metadata). Keep in mind that API support is not consistent for these and also depends on pixel format used.

• Border color and width.
• Wrapping modes.
• Min / mag filters.
• LOD min/max/bias.

[MarkCallow]

I have no objection to adding this metadata as which of these modes, etc. are best to use is clearly data dependent.

[MarkCallow]

CTTF TSG telecon 5/20/19. Keep this open for possible future addition. Won't be in first release. Can be added without bumping version as it is just metadata.

[Leadwerks]

Valve texture files support this information:
https://developer.valvesoftware.com/wiki/Valve_Texture_Format

The reason I am interested in this is because we have a 1:1 relationship in our engine between media and file paths. When the user loads an asset from a file path, it is cached with a weak pointer, and if the same file is loaded again, the original asset is returned. So the idea of having multiple different content with different settings loaded from the same file does not make sense in our usage.

Also, the fact that these properties are defined at creation time in Vulkan makes it more rigid, so it makes sense to define these as part of the texture file itself, for our purposes.

[MarkCallow]

I have no objection to adding a metadata item to provide sampler information. I want to hear opinions from more people before making a decision.

Note that since such metadata doesn't currently exist there will be much software that will ignore it. Also @Leadwerks note that there is nothing stopping you from adding your own metadata. Just don't use KTX or ktx as its prefix.

[Leadwerks]

Yeah, I can only say what our own usage is. I am curious to hear how other people are handling this. In glTF these properties are stored in the sampler, which to me implies the information is part of the model file, but my described usage has been a lot more practical for us.

Since the wrap and filter properties are optional in glTF, it makes sense that the values stored in the KTX files should be used if they are not present in the glTF sampler.

[lexaknyazev]

the wrap and filter properties are optional in glTF

The wrapping is always defined in glTF. By default, it is repeat when the sampler object or its wrapping parameters are not present. The filtering may be undefined but assets that require something other than all-linear should define their sampler configs. Besides, only supercompressed 2D textures are currently supported in glTF. KTX files on their own may be used in many other contexts.

[lexaknyazev]

The GPU sampler state properties may be grouped like this:

Wrapping

• up to three independent axes (U/V/W);
• normalized / unnormalized coordinates;
• five wrapping options:
  • repeat;
  • mirrored repeat;
  • clamp to edge;
  • mirrored clamp to edge;
  • clamp to border, requires border color:
    • predefined or custom value;
    • may rely on the texture's swizzle config and/or may have border-specific component mapping.

I don't think there's a strong case for standardizing on KTX metadata for these properties because the effective wrapping behavior heavily depends on the asset's texture coordinates that are not a part of KTX.

Filtering

• minification / magnification:
  • nearest;
  • linear;
  • cubic (optional);
• mipmap mode:
  • nearest;
  • linear;
• reduction mode:
  • average (default);
  • minimum;
  • maximum;
• comparison mode (depth only);
• LOD bounds: min, max, bias;
• anisotropy.

These options are usually specific to the texture data so it may be reasonable to store them inside KTX. There are two issues though:

• some options represent quality / performance choices that apps are supposed to make for themselves;
• not all combinations should be legal, e.g., enabling aniso with nearest filtering is undefined.