Bevy 0.12 introduced the ability to extend Materials.
If this is the first time you’re hearing about Materials in Bevy, then Materials in Bevy are effectively shader programs written in wgsl paired with the data they need to function. This means that you can write GPU programs to create almost any effect you’ve seen in your favorite video game.
The driving force for this feature was born out of a desire to extend Bevy’s StandardMaterial, so it makes sense to ask: What is StandardMaterial and why do we care so much about extending it?
StandardMaterial
Bevy’s StandardMaterial is full of complex behavior that’s already been implemented for you. From choosing between a base color and a texture, deciding how metallic or reflective the surface is, applying normal, parallax, or depth maps, and new in Bevy 0.12, dictating how light should pass through the object.
With all of this functionality, sometimes it makes a lot more sense to keep it around and write a little bit of code to allow what exists to keep working.
StandardMaterial implements a trait called Material, and new in Bevy 0.12, we have a trait called MaterialExtension that allows us to extend Materials.
We’ve got two examples to go through here, one from the Bevy repo itself and another I’ve written.
Bevy Example
The Bevy example is called extended_material in the Bevy examples folder. It takes advantage of the new MaterialExtension trait to extend the StandardMaterial with a quantized color output. The word “quantized” means to restrict the number of possible values, so you can think of this as a sort of “toon shader” that restricts the possible colors on an object resulting in a banding effect.
Now we could totally write a toon shader ourselves but the power of MaterialExtension really comes into play here because the StandardMaterial already knows how to apply lighting. So by extending the StandardMaterial we don’t have to implement integration with Bevy’s lighting systems.
First, like any regular Material we’d create, we have to define what new data we care about. This data will get passed to our shader.
Data in shaders gets passed via bind groups that are indexed, so bind group 0 is data Bevy controls relating to the view, and bind group 1 is our material’s data.
Continuing on with that through, our data has a group index and a binding index. The StandardMaterial uses a series of indices for the uniform data as well as textures and samplers. Those indices start at binding index 0, so we place our MyExtension uniform index at 100 so that we don’t overlap with the indices StandardMaterial has defined.
Then we define our data, which is a quantize_steps with a type of u32.
#[derive(Asset, AsBindGroup, TypePath, Debug, Clone)]
struct MyExtension {
// We need to ensure that the bindings of the base material and the extension do not conflict,
// so we start from binding slot 100, leaving slots 0-99 for the base material.
#[uniform(100)]
quantize_steps: u32,
}
With this data set up, we can implement MaterialExtension. This is where we specify which shaders we’re going to override. By default we’ll end up using the shaders the StandardMaterial implementation of Material defined, but we override the fragment shader and the deferred fragment shader here.
note: we have two overrides because the extended_material shader can run in forward rendering mode or deferred rendering mode. The StandardMaterial and this extension support both methods of rendering but we’ll only use one at a time, typically defined by the Bevy app itself.
impl MaterialExtension for MyExtension {
fn fragment_shader() -> ShaderRef {
"shaders/extended_material.wgsl".into()
}
fn deferred_fragment_shader() -> ShaderRef {
"shaders/extended_material.wgsl".into()
}
}
Making use of this Material is basically the same as using any Material.
Your query must get mutable access to the assets database for your extension and the material its extending.
mut materials: ResMut<Assets<ExtendedMaterial<StandardMaterial, MyExtension>>>,
and then when spawning in a mesh, we can apply the material. Note that here we use ExtendedMaterial with a base of the StandardMaterial and an extension of our newly defined MyExtension.
commands.spawn(MaterialMeshBundle {
mesh: meshes.add(
Mesh::try_from(shape::Icosphere {
radius: 1.0,
subdivisions: 5,
})
.unwrap(),
),
transform: Transform::from_xyz(0.0, 0.5, 0.0),
material: materials.add(ExtendedMaterial {
base: StandardMaterial {
base_color: Color::RED,
// can be used in forward or deferred mode.
opaque_render_method: OpaqueRendererMethod::Auto,
// in deferred mode, only the PbrInput can be modified (uvs, color and other material properties),
// in forward mode, the output can also be modified after lighting is applied.
// see the fragment shader `extended_material.wgsl` for more info.
// Note: to run in deferred mode, you must also add a `DeferredPrepass` component to the camera and either
// change the above to `OpaqueRendererMethod::Deferred` or add the `DefaultOpaqueRendererMethod` resource.
..Default::default()
},
extension: MyExtension { quantize_steps: 3 },
}),
..default()
});
With the Rust code set up, we can take a look at our shader.
Receiving the data we specified is just like any other Material, with the addition of our binding index starting at 100.
struct MyExtendedMaterial {
quantize_steps: u32,
}
@group(1) @binding(100)
var<uniform> my_extended_material: MyExtendedMaterial;
With this data we want to effectively use the StandardMaterial shader behavior, while adding in our own behavior in the middle to quantize the colors.
@fragment
fn fragment(
in: VertexOutput,
@builtin(front_facing) is_front: bool,
) -> FragmentOutput {
// generate a PbrInput struct from the StandardMaterial bindings
var pbr_input = pbr_input_from_standard_material(in, is_front);
There’s a number of different functions we can use to interact with the base material. In this case we’re using pbr_input_from_standard_material which does a lot of the access for us.
That gives up a PbrInput, which includes our StandardMaterial data as well as some calculated fields.
From here on our we can do whatever we want, but we want to apply the quantize effect in between when the material is lit and when the post-processing is applied.
We do this by taking advantage of apply_pbr_lighting and main_pass_post_lighting_processing.
and that’s it, we’ve used the StandardMaterial and our own data to modify the output of a regular StandardMaterial shader.
note: we’ve skipped over deferred rendering, which is supported in this example.
Dissolve Shader
A very similar example is this dissolve shader. I used to copy/paste the StandardMaterial code into this shader and my Rust code, which was a huge maintenance headache, so let’s see what’s changed.
We’ve got the same extension mechanism as last time here.
use bevy::{
pbr::MaterialExtension, prelude::*, reflect::TypePath,
render::render_resource::*,
};
#[derive(Asset, AsBindGroup, TypePath, Debug, Clone)]
pub struct DissolveExtension {}
impl MaterialExtension for DissolveExtension {
fn prepass_fragment_shader() -> ShaderRef {
"shaders/dissolve_material_prepass.wgsl".into()
}
fn fragment_shader() -> ShaderRef {
"shaders/dissolve_material.wgsl".into()
}
}
In this case I didn’t need any additional data, although I probably will add some in the future now that its less of a pain to maintain.
We’re also overriding two different shaders: the fragment shader and the prepass fragment shader.
The beginning of the prepass shader is basically copy/pasted from the default StandardMaterial prepass shader. Then I removed some things I didn’t need.
At the bottom, we write our dissolve logic. The logic here is solely for the purpose of defining the gaps in our dissolving sphere and discarding. So we return the same out variable that would normally get returned, but before that we do our calculation and optionally discard if we should.
The prepass output is used when determining shadows, so we have to put this logic here if we want the fragments of dissolving sphere to properly be reflected in the lighting.
var noise_step = 4.0;
let c = in.color.xyz;
let noise = simplex_noise_3d(c * noise_step);
let threshold = sin(globals.time);
let alpha = step(noise, threshold);
if alpha == 0.00 { discard; };
Then the regular fragment shader starts off just like our last example.
@fragment
fn fragment(
in: VertexOutput,
@builtin(front_facing) is_front: bool
) -> @location(0) vec4<f32> {
// generate a PbrInput struct from the StandardMaterial bindings
var pbr_input = pbr_input_from_standard_material(in, is_front);
var out: FragmentOutput;
// apply lighting
out.color = apply_pbr_lighting(pbr_input);
// we can optionally modify the lit color before post-processing is applied
out.color = out.color;
// apply in-shader post processing (fog, alpha-premultiply, and also tonemapping, debanding if the camera is non-hdr)
// note this does not include fullscreen postprocessing effects like bloom.
out.color = main_pass_post_lighting_processing(pbr_input, out.color);
Then we take the color the pbr functions calculated, and decide which areas were supposed to get discarded. We also define a border color for those areas that fades from the pbr color to that border color.
The alpha calculation matches our prepass segments.
// we can optionally modify the final result here
// custom code
var base_color = out.color;
var output_color: vec4f;
var noise_step = 4.0;
var noise = simplex_noise_3d(in.color.xyz * noise_step);
var threshold = sin(globals.time);
var alpha = step(noise, threshold);
var edge_color = vec3<f32>(0.0, 1.0, 0.8);
var border_step = smoothstep(threshold - 0.2, threshold + 0.2, noise);
var dissolve_border = edge_color.xyz * border_step;
output_color = vec4<f32>(
base_color.xyz + dissolve_border.xyz,
alpha
);
if output_color.a == 0.0 { discard; } else {
return output_color;
}
return output_color;
and that’s it, we are able to use the textures that StandardMaterial implements without having to re-implement the shaders.
Extending Custom Materials
Finally, can you extend custom materials?
Well, yes.
The docs for ExtendedMaterial specify that this works in the same way for any combination of base and extension.
So that’s extending materials in Bevy 0.12. I’m super excited for it to be easier to extend the StandardMaterial specifically, but I’m also on the lookout for new use cases now that we can extend our own materials as well!