Unpack normal maps using Z reconstruction.

[castano]

This PR adds support for packed normal map textures (BC5/EAC_RG/RG) by doing Z reconstruction the same way Unity shaders do.

Solves issue #818

This contribution is sponsored by Ludicon.

[CLAassistant]

All committers have signed the CLA.

[atteneder]

Thanks for the PR!

Please have a look at my answer in #818

…the same way Unity shaders do.

That's only true in some cases. In Unity 6.0's UnityStandardUtils.cginc you'll find this:

half3 UnpackScaleNormalRGorAG(half4 packednormal, half bumpScale)
{
    #if defined(UNITY_NO_DXT5nm)
        half3 normal = packednormal.xyz * 2 - 1;
        #if (SHADER_TARGET >= 30)
            // SM2.0: instruction count limitation
            // SM2.0: normal scaler is not supported
            normal.xy *= bumpScale;
        #endif
        return normal;
    #elif defined(UNITY_ASTC_NORMALMAP_ENCODING)
        half3 normal;
        normal.xy = (packednormal.wy * 2 - 1);
        normal.z = sqrt(1.0 - saturate(dot(normal.xy, normal.xy)));
        normal.xy *= bumpScale;
        return normal;
    #else
        // This do the trick
        packednormal.x *= packednormal.w;

        half3 normal;
        normal.xy = (packednormal.xy * 2 - 1);
        #if (SHADER_TARGET >= 30)
            // SM2.0: instruction count limitation
            // SM2.0: normal scaler is not supported
            normal.xy *= bumpScale;
        #endif
        normal.z = sqrt(1.0 - saturate(dot(normal.xy, normal.xy)));
        return normal;
    #endif
}

As far as I can see normal.z only gets reconstructed when using ASTC encoding or DXT5nm format.

So my concern remains performance impact, which is difficult to measure across many platforms and devices.

I appreciate you bringing this forward. Maybe we can figure out a solution.