elm-webgl

elm-webgl

Andrey Kuzmin walks us through Elm's syntax for writing 3D shaders and using them in a type-safe way from Elm.

PublishedNovember 22, 2021

Episode#44

Guest: Andrey Kuzmin (github) (twitter)

elm-explorations/webgl
GLSL syntax
Two shaders: vertex shader and fragment shader
Uniforms are declarative bindings to Elm values. Type safe mapping
Simple cube example
elm-explorations/linear-algebra/latest
Projection matrices
elm-webgl-lessons
ianmackenzie/elm-3d-scene/latest/
Luca's listing of 3d Elm examples and resources
elm-3d-scene examples
elm-csg (Constructive solid geometry)
Elm Japan 3d SVG art
#gamedev and #webgl channels in Elm Slack
James Carlson's talk about communicating with the GPU from Elm using Futhark: Making Elm Talk to Your Personal Supercomputer
Andrey's elm-webgl-playground examples

Transcript

And today we've got Andrei Kuzmin joining us to finally talk about some 3D stuff in

So, Jeroen, this is our newbie episode.

Andrei, thank you so much for joining us.

Thank you, folks.

It's really nice to be able to talk about this.

Yeah, it's great having you.

I've seen so many conference talks where you show awesome 3D demos and everything,

but I see so many Elm conference talks and things make sense.

It's like, oh, types, okay, types can help with this, types can help with that.

This can be type safe.

And then 3D stuff, and it's just all over my head.

So now we've got you here to ask you all our newbie questions.

And hopefully, other people who don't understand 3D will come along the ride and understand

it as a result of our newbie questions.

Yeah, thank you.

I'm really glad to be able to explain all the things.

I'll do my best in this.

Yeah, no, I mean, you've got all this great knowledge.

And I think that a lot of things are, you know, there's the curse of knowledge where

you know these things, and it's easy to take for granted.

Actually, like going through some of the Elm WebGL documentation, I had this moment where

sometimes the questions that people ask me, you know, about like a package I maintain,

they're like, what is this?

What is a page module in Elm pages?

What is, you know, and, and I have to be reminded like, oh, yeah, that's not obvious.

And it's so hard to know what questions people are going to have when you've thought so much

about something and built this thing.

And I had that same feeling reading the WebGL documentation is talking about like meshes

And I'm like, I need someone to explain it to me like I'm five.

So here we are.

So maybe let's start with this.

If you want to do 3D in Elm, you usually use the Elm dash explorations slash WebGL package,

What is WebGL's part in all this?

Why do we use it?

And why is it needed?

That's a good question.

So the reason why this package is brought into Elm exploration is because there is some

native platform, JavaScript that implements the integration with the WebGL.

And there is also a part of WebGL that is implemented in the Elm compiler itself.

It's like it's really tightly coupled with the types that compiler generates.

So, so WebGL is like a language, like a DSL for expressing 3D things.

And Elm has like GLSL as a syntax you can do.

And it creates, it actually knows the type information about the GLSL snippets.

Are the things I just said accurate?

That's correct.

So it does type inference for the shaders that are written in GLSL.

That is a language that looks similar to, I guess, C.

So GLSL is the language and WebGL is the runtime.

WebGL is like a set of APIs, I would say.

To do like 3D graphics or any graphics that are accelerated by the GPU.

And those are natively supported by every browser out there?

Yes, pretty much.

They are like WebGL one that like the first specification that we are relying on is I

think you can safely assume it runs in the majority of devices like even mobile phones

like Android or iOS.

So okay, let's break down every single term.

And Jeroen and I are experts at not knowing what these terms are.

I'm so good at that.

We're really good at that.

What is a shader?

Well, a shader is like a block of code with a main function.

And there are two types of shaders in the implementation.

One is called the vertex shader and another one is called fragment shader.

And these are the definitions of like the code that runs on GPU.

So they are being compiled and executed on the GPU.

And then all the data that you pass just goes through them.

And as the result, you get some visual things on the screen.

I guess it's like a very...

Like maybe it's an oversimplification.

Like if I think of shading, I think of like maybe like coloring something in with a crayon

Like what's the metaphor of shading?

Like what does that mean a shader?

I don't think...

I don't know an answer to this question.

But what is it at a high level?

Like if you were to describe what a shader is doing at a high level, not with code, what's

To describe what?

I think for the purpose of like explaining how things work in the WebGL, I think it makes

sense to maybe talk about fragment shaders, vertex shaders and fragment shaders separately.

So if I was to explain what the vertex shader is, it's a shader that runs for every vertex

and it produces an output of it is where this vertex can be found on the screen.

Is a vertex the triangle or the point?

It's triangles?

So, well, depending on the geometry, if the geometry is triangles, then the vertex is

a corner, like a point on that triangle, a corner point.

It can also be aligned.

So then it's like a point, like an end of a line segment.

So it's defining how you draw the points, but it's not defining what those points are?

The input to it is attributes.

And attributes is in elements like a record with user defined fields.

It's an information that you can attach to it.

So it's like, it's very generic.

You can say, oh, you can attach coordinates to it, or you can attach color or texture

So any information you find useful, it can be attached to it.

And then this vertex shader is going to transform that into coordinates on the screen plus some

output data, which is called a variance.

So would a shader be almost similar to like CSS rules that could define, you know, a color

or like a background image to put over something or a transformation to like move something?

If you think of the composition of two shaders, then maybe you can think of it in this way.

So if let's say if the input to the vertex shader is like a mesh that is basically, well,

if you talk in terms of triangles, like a triangle or geometry, it's a bunch of triangles

and each point of this triangle has user defined information like color coordinates.

Then this goes through the vertex shader and it becomes, for each of them, you know where

it is on the screen plus any other information that is then interpolated.

When another shader picks it up, it takes, it operates on fragments, but you can think

of them as pixels on the screen.

So it basically, it fills those triangles and what it gets as an input is interpolated

So if you have a triangle that has like red, green and blue attached to its vertices, then

if you use that in the fragment shader and the fragment shader determines the output

color of each pixel, each fragment, then you might just like interpolate it.

So if you return this output, this varying from the vertex shader, then the triangle

is going to be filled with like interpolated color depending on how close that each point

is to either of the corners.

So okay, so there's this GLSL that you can put inside of the body of Elm code with these

little notation square brackets, GLSL pipe, and then you actually input GLSL code.

And then Elm infers type information about that, right?

So like what's the mapping between the GLSL, what is it taking from the GLSL and between

that and the Elm type?

So you define, when you do a GLSL snippet in Elm, it gives you a shader type, right?

And then what does it map the GLSL types to the Elm types with?

So usually at the top of the shader, you would have some global definitions of attributes,

uniforms, variances for the shader, for the vertex shader.

That would mean that it builds up a shader that has all the attributes are put together

into a record type.

All uniforms are grouped in the record type and all the variances are grouped in the record

Like I forgot to mention what the uniforms are.

The uniforms are some global constants.

So they're not part of the mesh, they are part of the rendering.

So like camera position.

So that's basically like an input that you can use to, that allows you to, through Elm,

change the camera position.

You can like make that vary now and you've basically created a binding between Elm and

the WebGL interface where you can move the camera position, for example.

It's all passed in the declarative way, but if you use like animation frame subscription

and you pass a different camera position, then you'll have like a moving camera.

That's very cool.

So you're right.

So it's this declarative thing where you have your uniforms, which are these like global

values in WebGL and you declare them in your GLSL snippet and then you pass in types, which

are Elm is going to check that the types match between the uniforms you've declared in your

GLSL and the ones you use in your Elm WebGL code.

And then you can change those values and the camera moves or the color can change or the

rotation can change.

And this type safety is not just for uniforms.

So basically you have two shaders, their type signatures have to align and also you have

mesh that is defined by its vertices, like attributes, attributes type attached to every

vertices and those attributes will have to match with the input to the vertex shader.

The uniforms have to match between the vertex shader and the fragment shader and the uniforms

that you pass, like the uniforms that you pass.

And also the variants coming from vertex shader into fragment shader, those values that are

interpolated, they also have to match.

So Elm verifies that all the types align together and it really helps to avoid some problems

that you might otherwise encounter when doing it in a raw JavaScript.

Or just the C language that is GLSL.

Or you still have to GLSL if you want to use the raw Elm WebGL.

Is that part of GLSL, does that give you some of that type safety in itself but then Elm

gives you the bindings to the uniforms that it passes in?

Or you're saying if you use a JavaScript library...

If you were to do it yourself in JavaScript, then you would be getting runtime exceptions

that you'll have to...

So you'll have to run your code in order to see them.

Does it give it to you as soon as you start the GLSL and then it says, oh, the types are

Does it compile the GLSL at runtime?

There is a bunch of operations you need to do in order to get it working.

There is a pretty big boilerplate, I guess.

You have to compile the shaders that may fail.

Then you have to link the two shaders into a program.

So you need to take Vertex Shader and Fragment Shader, link it to a program.

And then when you perform drawing operations, you have to correctly set all the buffers

and pass it to the program and run it.

So that may fail.

And the uniforms may not line up from being passed in from JavaScript and being run in

And when things go wrong, it's a runtime exception.

And usually you see just a white screen with some console errors.

I think I'm starting to get a picture of why people get very passionate, it seems, about

doing 3D stuff in Elm.

Because it sounds like Elm has really...

It's so interesting to me because I feel like I have my finger on the pulse of Elm stuff

a fair amount, except this stuff is such a blind spot for me.

But I see people getting really passionate about it.

And this must be one of the reasons.

I know some people who even used some raw WebGL before trying out Elm and they say,

wow, this is so much easier because of all these guarantees.

How are the compiler errors?

When you mess up either the GLSL or when you mess up the linking, are the compiler errors

You mean coming out from the raw WebGL or the Elm ones?

I mean when you...

When you're using WebGL in Elm.

Yeah, when you use WebGL in Elm.

Yeah, I think they are.

It will be basically like, I want this type, but you pass this type.

So I'm curious to know, and I'm wondering whether this is a good way to start a conversation

because when you need to do something in 3D, how do you think?

What do you start with?

Do you start with writing a bunch of vertexes and attributes or what is your process?

Like maybe we need to define what that something is because it may depend on that.

Like, so can you think of an example?

Let's say you want to draw a cube or a triangle.

So well, then the first thing I would think of is like, I need to define a mesh for the

So I need to visualize it in 3D and I need to write down coordinates for all the triangles

that form the cube.

So the mesh is just the, I want to say the physical representation of the 3D object,

just like the box, like where are the coordinates of every point?

That is a mesh, right?

In the simple form, yes.

But like what you say coordinate is, it's an attribute.

It's a record where you may have coordinates and you may have something else, whatever

you want to attach to it.

And a mesh is a list of, how do you say, triplet, like a tuple of three.

So it's a list of those.

And each of the components of that triplet is this record type that you define yourself.

Which often contains the position or the original position and then some additional information

I see that in this like simple cube example, there's like right front top, left front

top, left back top, et cetera, et cetera, for all the three dimensional points that

you would map out for a cube.

And then it's creating a face that is stretching from each of those with these three point

Like when constructing a cube, I think you can be like more clever.

Maybe you define one, maybe define a square.

And then if you know like how to transform that, how to do like a rotation, maybe then

you can define all the sides of the cube.

I think there is in one of the examples, I think there was this kind of implementation.

Yeah, I can confirm.

Or you just write it down, like all the coordinates.

There are so many different skills in this world.

Like there's the geometry and these mathematical transformations and the textures, the 3D stuff.

That's really interesting.

So if we talk about textures, that is another thing in WebGL.

So let's say if you want to have a textured cube, then you might want to assign texture

coordinates to each of the face, which means basically where this vertex can be found on

a texture, on an image.

And then you can use this information in the fragment shader that determines the output

color and you can pass the texture as a uniform and then you can do a lookup using this coordinate.

You can find where that pixel is and take its color and return it.

Then you get like a textured rendering of a cube.

And that's how we apply a texture on a face or...

So you have your mesh for your cube.

What happens next?

So then you need to think of the scene.

You need to set up a scene.

So you probably think of, do you need to move the cube around?

Do you want to have...

Where do you position the camera?

Where the camera is looking at?

And in the role of WebGL API, that would be all matrices that define this.

So you may want to have like a transformation matrix that you want to apply to your cube.

If you define a cube in its own local coordinate system, like I don't know, based on zero,

Then you might want to move it around or if you have many cubes and you may have, you

may position them in 3D.

That is defined by matrix that transforms from local into world coordinate system.

Is there any convention with like the Y axis going downwards as it grows positive or does

it grow up as it grows positive?

There's like, usually I think I would use that up.

So the intuitive thing.

X to the right and Y from the camera.

I think this is what I would use.

But I think you're not really, you're not restricted because you can do all these transformations

for the matrices.

Like you can flip the axis, you can do whatever you want.

And you could create some mapping of real world units.

Like I've heard Ian McKenzie talk about trying to use real world units to model things.

Yeah, certainly.

So this is the first matrix, let's say the matrix that moves around objects in the world,

like positions them, rotates them.

You can compose, you can combine all this transformation in one matrix basically by

like first rotating and then moving.

I feel like I'm missing something.

So you have your mesh, you say that it's at position 0, 0, 0, and then you first set a

No, like first you may want to move object around.

If you want it to be positioned at 0, 0, 0 and just render it, then you don't need that.

Then all you need is like a camera transformation.

So that is like there is another package that you have to use, which is exploration linear

algebra and that provides all the useful functions to define like...

To like manipulate matrices basically, right?

Yeah, and create some like various matrices that you might need, like a projection, like

a perspective projection matrix.

So what is a perspective matrix?

So there are two types of projection matrices that exist in linear algebra.

One is the matrix that like it takes your 3D thing and it projects it onto like a plane.

Yes, that's a definition of projection and what perspective projection does is it makes

it so that like parallel lines intersect.

If you prolong them, then they would intersect.

So you get this kind of realistic view and orthographic projection keeps the parallel

lines parallel.

Okay, so it's so that you can imitate seeing from an eye.

So like a lens or different kind of lens?

Like would you imitate like a fish eye lens versus a telephoto lens type of thing with

those transformations or is that different?

I think like one of the inputs is an angle of view as far as I remember.

So you might want to like you might be able to do this, but I'm not really sure about

the lens transformations whether you can achieve them using the existing functionality.

Okay so we were talking about meshes and just to reiterate like a mesh is just part of a

So when you define your GLSL snippet in ELM, you're defining a mapping, you're defining

your mesh which are the points, the vertices.

You're defining your uniforms which are essentially the parameters that you're binding between

ELM and GLSL that you can pass in things that vary like camera position.

You're defining them, you're saying I want to have those uniforms, I want those attributes,

but you're passing them in from ELM, right?

Right, like it seems like it's almost like your model, right, for your GLSL.

So attributes is mesh and then uniforms is just like an ELM record and then you also

get two shaders and then you put it all together into what is called an entity in the ELM Web

Then you have a list of those that you pass into the top level function that creates a

canvas element and like a DOM element so that becomes HTML.

And so like this whole GLSL thing is the purpose of it is to like it's a language that is compiling

down to highly performant stuff that's running on the GPU of your machine and so these like

uniforms are I mean it's compiling down to like very fast GPU instructions essentially,

And so ELM is able to just sort of vary these few pieces but then it just runs very fast

GPU instructions from that, right?

Yes, there are numerous performance optimizations in this pipeline.

So first of all, all the meshes are cached so whenever this like a WebGL to HTML function,

whenever it receives a new mesh is like in one of the entities, it would upload it into

a GPU memory and cache it.

So when you call the same function, you pass the same mesh, it doesn't have to reupload

the data to GPU.

How does it detect whether it's the same?

Is it by reference or does it do an equality check?

That's a good question.

So I think in the past what used to be is the JavaScript object is being tagged with

an ID but I think what we switched it recently to is using weak map in JavaScript if you

know about this.

And the overall cache kind of or like a representation of what is cached, what's not is a part of

the virtual DOM node that is created for that canvas, for the WebGL canvas element.

And this node, it stores things like uploaded, like references to uploaded textures, like

all the buffers for attributes but the uniforms are not.

So the uniforms are changing from like you can set some different uniforms.

That's the whole purpose of them to be able to move around and the uniforms is usually

not that much data when compared to like a mesh that defines a very complex 3D object.

Yes, so that's why you try to have the uniforms change but not the meshes, right?

If you change your mesh 60 times a second, then you will have to upload it onto GPU every

time and that would make it slow.

So yes, this is like it's a really cool performance optimization but it's also it can be thought

of as a limitation like if you want to be changing the mesh instead of moving it around.

So you need to think of how you can achieve what you want by not changing the mesh but

by I don't know having multiple meshes and like positioning them in the way you want.

Then like varying your uniforms for example, right?

That would be because that's just running the same mesh that it's computed and it knows

the ways it varies based on those uniforms.

But then at the same time, you don't want to have too many of them.

Like you don't want to have too many entities.

Because each entity internally results in a draw call and a draw call is like this final

execution of all the things together and it's on overhead.

So any call to GPU has some sort of latency, some overhead.

Like even internally what we do is we try to keep the calls or what like the settings

or even like uniforms we try to keep what the previous ones were that were set on the

And if a new operation comes, we see whether it's the same as it was set.

Basically some sort of diffusing but like a very simple one.

And we do not do it if it is the same to reduce the number of calls to WebGL.

You prefer to do an equality check before checking whether you need to run before rendering.

So let's go back to having small uniforms and meshes that don't change.

Let's imagine I want to draw a hand and I have one mesh per finger and one mesh for

the entire hand and I want to move the fingers.

Do I have a uniform that defines like the rotation of the finger or do I have like one

mesh for the entire hand?

That's an interesting question.

I haven't done this but I can think of how it's possible to do it.

I think what you sort of need is you need to define a skeleton of some sort.

And if you think that the origin position is of the hand, then I guess you can do like

the next like bone or something you can specify it in the coordinate system of the hand.

And then you can have some sort of tree built up that has like relative positions in it.

And then if you move the hand, then everything else will move with it.

But then what you need to do internally, you would need to flatten out all the transformations

because in the end you're rendering a list of things, right?

So every bone will have to be an element in that list.

So you would need to compose its transformation with the positional transformation of the

Which you would do using matrices?

That's the raw WebGL approach because in the end you pass like a matrix 4 and in the end

you use this like matrix multiplication on the GPU.

But there are other ways to model things like using a library called Elm Geometry from Ian

McKenzie which is a completely different way of thinking of transformations.

There are types like frame 3D and that type gives you the possibility to transform between

two coordinate systems.

But at the end of the day GPUs only understand matrices, right?

Like that's basically…

So the way down when you want to render you would have to convert frame 3D into matrix

To speak the language of the GPU.

So we've got our like our shader for our cube and we've got our faces defined which is our

cube mesh, right?

Our mesh is all of the faces of this cube.

And then we create an entity from that mesh which defines the faces of the cube and the

shader which…

So what are those two shaders?

So the vertex shader and the fragment shader.

The vertex shader would work, would be called for each of the vertices and it will be passed

attributes like that are that we attached to all the points that define those triangles

from the mesh and the output is where things can be found on the screen in the clip space

So it's like from minus one to one on all axis.

So it's not like pixels coordinate.

And then the fragment shader runs and it runs for all the pixels within each triangle and

it returns a color which is RGBA and it's defined as also like from zero to one for

each component.

So R from zero to one, B from zero to one.

So the fragment shader runs for every pixel that would be seen on the screen?

I mean it's not computing anything that wouldn't be seen on the screen because it's behind

the camera or something.

Yeah, so what like there are things.

So if a thing is outside of the visible area, I think there are different ways like you

can get rid of back facing triangles if you, okay, that's a, let me think a bit more.

I'll explain this.

When you define the projection matrix, you need to pass two values, which is the nearest

clipping plane and the furthest clipping plane.

Everything that falls out of this will not be visible.

If something is too close to the eye, then you don't show it.

And what you can do in order to get things invisible is the output is like, so the output

from the vertex shader as X, Y and Z, the Z coordinate, it doesn't affect into where

the pixel is positioned because our screen is 2D, but it's still, you can still set like

a value that is less than minus one or more than one.

And that will mean that it's not going to be rendered, this vertex.

And I think this is what the projection matrix would do.

It will like map coordinates to being outside of the so called clip space, this visible

part of coordinates that are returned.

Like the range of coordinates that are considered visible when for the output of the vertex

And then another way of making things invisible is you can rely on the winding of the vertices.

You can rely on what?

So you have like counterclockwise or clockwise coordinate, like positions of the vertices.

And then if you create an entity with settings, you can make it so that it only renders like

front facing or back facing triangles.

Is that in order to not have to compute too many things?

Is that just a performance optimization?

Like you want to draw a house, but you don't want to draw all the walls behind it.

So it's just a facet.

Same for the cube, I guess you don't want to render the inside of the cube.

You only want to render the outside if it's like sealed.

You can't look inside.

So I think this is kind of starting to click for me.

So like, okay, so we're creating an entity using WebGL.entity and we give it a vertex

shader, which we've talked about.

We give it a fragment shader, which is telling it how to color the faces of the cube.

And if that's just a shader, so it takes a uniform, we could have the uniform could allow

us to control the color.

It could allow us to vary the color and bind that from Elm.

So if we wanted to, we could just say, depending on the face, give it a fixed color, or we

could give it a gradient color if we wanted to create that mapping.

And then we give it a mesh.

Yes, you can do coloring from uniforms too, or you can have them coming from the mesh

and pass as varying from the vertex shader into the fragment shader.

You have all the options.

I'm guessing the fragment shader is mostly used for lighting.

So you tell us where the sun is and then it makes some pixels lighter or darker based

on their position.

Yes, you can use it for this too.

Like if for lighting, you would need to have a normal to the surface.

So a normal, it really depends on whether your object is smooth or not.

Let's say our cube is not smooth.

For that, I think you can even calculate the normal in the fragment shader, but you can

also have it attached to each of the vertices when you're defining the cube.

What is a normal?

It's like a unit vector from the surface, like going to the outside from the surface.

Okay, so it's like...

Pointing where the surface is facing.

And then you have another, like you have a direction of the light or like this kind of

If you have directional light, like the simplest possible is like the simple flat shading where

you have the direction of the light.

And then when you're trying to determine what the color of the pixel should be, you can

check how these two vectors align.

Like if they are facing each other exactly, then I guess it will be highlighted.

But if they are facing in the opposite direction, sorry, if they are facing in the same direction,

then it's going to be in the dark.

Yeah, because you're pointing, the light is shining on the backside of the cube or of

Then it means it won't be shown.

Okay, so this is cool.

So like the the variances, so this is like the sort of third piece of a shader.

It can have a varying which is an output from the shader.

Yeah, so a vertex shader has a variance as an output.

The fragment shader has them as an input.

So for example, if you have an output of the color given which face of the cube we're on,

then that varying output from the vertex shader would be, that color would be an input to

the fragment shader and then it would just take that color and possibly transform it,

but basically paint that color.

Yes, basically.

It doesn't take it exactly right because you set it onto three vertices of a triangle.

Right, remember the vertex shader is working with only the corner points, whilst the fragment

shader is filling the whole triangle.

So what you get there is an interpolation, an interpolated result based on how close

you are to each of the vertices.

So like if it's very close, if it's at the vertex itself, then it's going to be the color

that is specified for that vertex.

If it's somewhere in the center and you have specified different colors for all of the

three vertices, then it's going to be like an average of colors.

Can you determine how that interpolation is made?

Can you use different interpolation functions or is it always the same?

It's the same and I can't tell how it works.

I think it's like linear.

And I'm guessing if you want to be more precise in how the values change from one vertex to

another, then you need to multiply the number of triangles, right?

So if you have, so let's say your cube is really smooth, so let's say your cube has

a very uniform color, then you draw two triangles for one face.

But if you want it to be a lot smoother, then you can cut it up into more triangles and

have the colors change slightly more.

Yeah, if you want to achieve some weird coloring that is not a simple linear interpolation,

then you can have, I guess, more segments to curve your own gradient.

Not sure how that would work.

Maybe you'd rather have a texture and then you map it to all the four vertices of each

Yeah, or if it's not a cube, but it's more bumpy, then you need to make a lot of triangles

And then the lighting can have different effects on each triangle separate from another.

Whereas if you have only two triangles for an entire face, then what you can do with

it becomes very simple.

Too simple, maybe?

For smooth surfaces, it's like you wouldn't set the normals the same way as you would

do for the cube.

For the smooth surface, the normal would be pointing from...

It's sort of like as an average of all the normals of all the triangles that start with

Let me think of how to say it better.

If you have a point on such a bump, you might have multiple triangles around this point

and they will all have their normals as the strict normals pointing from this triangle.

But what you want to attach to that vertex is not either of them, but something in between

to have a smooth normal.

And then those normals can be passed as varyings and then the normals themselves are going

to be interpolated.

So what you will be getting in the fragment shader is a normal that is smooth.

Because if you use normals for a cube, flat shading, then you will see the rendering,

the output will be looking very low poly.

All the edges will be very visible because the normals are going to be jumping from one

triangle to another.

It's not going to be smooth.

So the lighting is not going to be smooth because of that.

But it really depends on what you are trying to achieve, what kind of look.

Yeah, not a cube at this point.

So let's say I want to draw the normals or to have the normals be sent in the varyings

from the vertex shader to the fragment shader.

How do I go about finding GLSL documentation on how to do that?

Because I need to write the shader, right?

That is not Elm code.

So where do I find documentation?

How do I get started to do that?

I learned about this by looking at some examples.

So there are general examples of how to use WebGL and all the GLSL code is the same there.

You can look it up there or you can even find some examples in Elm that are pretty simple.

Yeah, with this normal thing, I think it's slightly more complex than just rendering

If you want to get into lighting, then it gets more complicated.

Maybe at this point, you want to think of like, do you really want to invest a lot into

learning 3D graphics in WebGL?

Or maybe you'd rather switch to some of the existing packages like Elm 3D scene that would

You can set up a scene with lights and you'll get nice physically based rendering out of

So GLSL is a special syntax in Elm, but can you construct it programmatically?

Can you create one from a string or something?

It's like statically compiled.

Okay, so you can't have a package that creates...

So you can't have an Elm package that creates a GLSL function programmatically based on

user defined inputs?

No, there used to be unsafe API, but you know that in this case, you have a string and you

pass that string into runtime and you interpret that.

So you would lose all the guarantees.

So at some point this was removed and this thing is what is currently being explored

of how to have a way to nicely compose shaders or to be able to share some bits of code,

like a function that you can use in your shader that does something useful.

So I think that is currently being explored.

So is the main use case for the Elm WebGL stuff games?

I mean, games are the obvious one.

I know Ian McKenzie is very passionate about CAD type stuff like modeling physical objects.

What are the types of use cases people would use this for?

It's very broad, I think.

Like games, yes, or maybe even data visualization or anything that would require 3D visualization.

You can think of an advert or like a product page where you have something physical, like

I don't know, you're selling phone and you have like a 3D model on the page that lets

your users rotate it and look at it in 3D.

I think Ian even mentioned a spinner, you can have like a Nijax spinner rendered in

Yeah, I think is the kite project also in WebGL that Erkal Selman created?

I think it might be.

No, that is using SVG.

That's SVG, okay.

But Erkal has been doing some really cool visualizations using Elm 3D scene.

Data visualizations?

No, more like creative coding kind of thing.

Because with functional API, you get all the function, composition, all the things you

can produce generative art.

You can do random generators and yeah, very cool.

So what are some of the cool projects that people should take a look at to get inspiration

and look at some examples?

So one thing I would share is there is a very nice overview of almost all the 3D graphics

projects built in Elm.

Luca, I hope I pronounced his second name correctly, Luca Mognini put together.

It has all the games and art and all the crazy things built in Elm.

I think that is a really cool overview of just seeing what's possible.

There are more libraries.

There are other projects that are more like libraries.

So for example, Roman Potasov, his GitHub is just gook.

He implemented a WebGL playground.

He implemented a WebGL playground that is very similar to the package that Evan implemented,

but it uses WebGL under the hood.

A lot of things can be put together using that package and it's very simple.

Another thing that I would recommend to check is Ian's Elm 3D scene and all the examples

there because it's a high level API.

You don't need to deal with shaders.

You can put together 3D things using his libraries.

If you'd recommend someone to start with 3D in Elm, you would recommend checking out Elm

3D scene or would you recommend them to use Elm WebGL to start with?

I think you can start with Elm 3D scene.

I think it has a friendlier API, but it really depends on whether learning the basics also

has its advantages because you're more flexible in what you can do.

But if you have a use case that maps to what's possible in Elm 3D scene, then that's going

to be an easier path.

Another project that I wanted to highlight is a project from Tomas Latal.

I think he just recently made it public.

CSG stands for constructive solid geometry.

What this package allows you is you can construct 3D objects using primitive 3D objects and

Boolean operations between them.

It's very early stages.

It's work in progress, but there are some really cool examples.

You can make a chess piece with it using some cylinders.

It's easier to create 3D shapes using this method than writing down or constructing coordinates

If you have a cylinder, a sphere, a cube, then you can subtract them or group them and

then produce another shape using those operations.

Because you don't have those shapes in WebGL, everything is a triangle.

There are no shapes in WebGL.

There are shapes in Ian's package.

There is a project that I created that lets you load OBJ files.

You can draw things in Blender using 3D graphics software, like 3D design software, and then

output an OBJ file and then load it into Elm.

That opens a lot of other possibilities for creativity.

You can prepare your game assets and then load them into Elm.

Using ports, decoders, and stuff like that?

It comes with this package, what is it called, an Elm HTTP plugin or this decoder.

You can plug into Elm HTTP and it would load and pass it to you as a triangular mesh, which

is a type that is supported in Elm 3D scene.

You get it straight into the format that it expects and then you can render things using

Ian's Elm 3D scene.

I have plenty of examples like falling ducks, for example.

That's awesome.

Yeah, Luca's Medium blog post is really cool with all these demos.

People should definitely check that out to get a sense of what people are doing with

I remember when Elm Japan was announced, which is a little sad to talk about since it got

cancelled for obvious reasons.

But I remember the website was so cool.

I wonder if it's just this little floating Tokyo city and cityscape with cranes moving

around and you can see it floating in the air.

I think you could change it from night to day or something like that.

I love these delightful little visualizations for conference websites and stuff.

It really makes things fun.

I wonder if that was created with Blender or with raw WebGL or what?

As far as I understand, I think it's not really using WebGL for this.

I think it's using SVGs.

But what you can actually do is you can render 3D things using SVG.

You can basically do all the projections and all the coordinate transformations yourself

There are packages that you can use in order to do this sort of thing.

I think the main problem with this is that you will have to deal with the Z index yourself,

with the Z buffer yourself.

You have to render one thing after another thing in order if it's closer to the camera

or intersections would not be possible easily.

Yeah, it does say it's created with SVG.

That's really fascinating because it sure looks like a nice 3D model there.

So all these matrices that do the coordinate transformation, you can multiply the coordinates

by these matrices to transform them yourself and then render the final output, the final

result using SVG.

It might be slower in some cases, but if it's a simple illustration like this, a simple

drawing like this, then it…

When you use Elm Explorations WebGL, it renders using a canvas, right?

An HTML canvas.

So WebGL to HTML is the function that puts this DOM node that is a canvas, that ends

up being a canvas element.

And you can even pass attributes to it.

Yeah, regular HTML attributes.

Yeah, like a width and height style.

What's the best place for people to ask questions if they're getting started with this stuff?

Is the game jam…

I know there's a pretty active game jam channel in the Elm Slack.

Is that a good place for people to ask?

Yeah, the game dev.

Oh, there's game dev too and then there's the game jams, right?

No, I think it's the only channel.

We used to run game jams.

This is where we announced them, but the channel is called game dev.

And then on top of that, there is the WebGL channel.

That might be more appropriate if you're not into games, but some people ask questions

in both and it's totally fine.

Yeah, there's this whole hidden community that's been off my radar, but it looks like

No Dillon, don't start running games.

You have tools to make.

Actually games are how I got into WebGL in the first place.

We were working on a game and that was slow and we tried to render it with WebGL.

That was using WebGL in 2D.

You can do 2D, it's not restricting you.

And then I thought, maybe it would be cool to do something in 3D too.

And I created the Rubik's cube game.

And then after that, okay, well, where can I go from this?

Maybe I can do a physically simulated dice.

And things got complicated.

You can layer so many things into it.

Physics is a whole other level.

Creating things in 3D is one thing, but having bodies interact with real world physics.

It seems like a lot of game development is done using Unity and these sort of game engines

that do abstract away a lot of the details, I imagine.

But yeah, physics.

It was quite the ride getting it to work.

I didn't start from scratch.

I looked at the Canon JS, but that is a library in JavaScript that is very object oriented.

Oh, so it didn't help.

It did help a lot.

It's very clear how it's written.

So I was able to read and understand the code.

But a lot of algorithms, they are very imperative and converting them into a purely functional

style was quite a challenge.

Just out of curiosity, because apparently Elm is able to send things to the GPU.

Can you send computations to be done on the GPU in Elm?

Or is it just WebGL?

No, the current WebGL is only letting you render things.

And I think arguably the way Elm APIs are done, if there was a thing that would let

you run some simulations or computations on the GPU, I think it would probably have a

very different API from the graphics oriented API.

James Carlson gave a talk a while back about using Futhark to do GPU computations from

I think it was using ports.

Yeah, it's an interesting space.

I mean, a lot of things are changing right now.

So I think WebGPU is being rolled out in browsers, but not in all of them.

Yeah, WebGPU is a lower level API, the new API for interacting with GPU and that has

broader range of use cases.

It makes things possible that were not possible before.

But the current Elm API is based on WebGL 1.

And it's using the current pipeline has some limitations.

For example, what you cannot do is you cannot do multipass rendering, which is render this

thing on a texture and then take that texture and then use it in the subsequent rendering.

And this technique is used a lot to achieve things like shadow maps, which is a way to

render shadows, which is like a really clever way to do them.

So you render the scene from the light's point of view.

So then you will see what's highlighted, what's in shadows.

And then you render it from the camera point of view, but you use that as a map to figure

out what is in the shadows, what is highlighted.

Or things like HDR or things like, well, there are a lot of use cases that would require

And you're saying that this two pass system is not possible with WebGL?

With the current API of WebGL, you only do one pass.

Sorry, of WebGL or Elm WebGL?

And we are still exploring possible ways of how to achieve multipass rendering.

There are some ideas.

I think Ian has some ideas.

I did like a rough prototype of this.

And so I'd like to prove that this is possible, but the API of that is not good.

The most problematic thing about this is you have to allocate all these memories for these

intermediate buffers in order to store the intermediate renders.

And the API is declarative in a way.

You only have one way to pass things in.

You don't have a way to say when to remove something from the...

When to clear the allocation.

So you don't know whether you rendered this onto a texture.

You don't know how long are you going to need this texture.

Is it the result of this rendering?

So if it's some intermediate result that is still 60 frames per second, or it's like a

texture map, you render it once and then you're going to keep it forever and use in all the

subsequent renderings.

It's really hard to tell.

There are multiple use cases and coming up with an API to support all of that is very

Well, I feel like I've got a lot better grasp of 3D stuff in Elm.

So there's so much more to learn, but I feel like I've finally started to wrap my head

around some of the basics.

So this was great.

So what should people take a look at if they want to dive in more?

We've got the Elm Explorations WebGL package that we've been talking about.

The docs are very thorough.

So first read the docs.

Then check the examples.

So there are examples in this package, like on GitHub of this package.

I think some of them have been even published on the Elm blank website.

So you can see them there.

So yeah, play with those.

And then at some point you might need to learn a bit more underlining things.

And there's a learning WebGL and there is a GitHub repo of I think, Nacho Martin.

He tried to follow these lessons and do things in Elm.

So you can see his intermediate results of completing these lessons in Elm.

But if you don't want to, then Elm 3D scene, it has its own API and it also has a very

good documentation.

It even has screenshots of what you see if you use this setting.

And you've got your Elm WebGL playground repo.

This is what I used in order to play with the API and see how to do things.

Because the very first thing, if you want to do 2D thing, then how can you render a

sprite on the screen?

That's a very basic thing.

Just like a 2D rectangle with an image on it.

I have an example like this.

Yeah, and you've got lots of games.

You've given many conference talks about this stuff.

So if people want to see more advanced stuff, less fundamentals and intro to the basics

of WebGL, they should definitely check out your conference talks.

We'll link to some of those as well.

Or just talk to me directly.

On Soundscapes, on Elm Slack or on Twitter.

I'm really glad we got to sit you down and ask all of our dumbest questions.

So thank you for being patient with us and walking us through fundamentals.

And I hope some other 3D newbies found it useful as well.

Yeah, thank you.

Thank you very much for inviting me.

Very good to be useful.

It was a pleasure.

Thanks so much, Andrey.

And Jeroen, until next time.

Until next time.