elm-form

elm-form

We discuss the power of decoupling from data types using low-level data, and how dillonkearns/elm-form gives you simpler wiring that feels like magic but isn't.

PublishedApril 24, 2023

Episode#81

dillonkearns/elm-form
Exploring a New Form API Design episode (discussion of some ideas that became dillonkearns/elm-form)
Ellie example of elm-form
Meetdown code that does additional checks to ensure that opaque type are trusted (sending unparsed data in Lamdera sendToBackend avoids this pattern)
Lamdera example with elm-form
elm-app-url
dillonkearns/elm-package-starter

Transcript

Well, we talked about exploring a new API design for forms a while back, but since then

I would say I've perhaps formed some new opinions.

I just wanted to inform you.

That's very nice of you.

Informally, of course.

Okay, how am I supposed to react to this, man?

You don't need to mutate at all.

What's that mutation joke coming out of the blue?

You just said react, so I felt obliged to say something to defend Elm's honor.

Well, I guess in my view, this is how people should react.

You don't want to start a flame war or even start at the tiniest ember, right?

I just want to give people an update.

That's all I'm saying.

That sounds good.

So yeah, we did.

Moving along, we did have an in-depth discussion about designing a new form API.

And at the end of this episode, we'll link to the episode in the show notes, you asked

me if I was planning to create a standalone version of this form API.

At the time, I wasn't sure whether or if I could or would.

And a few more people asked me, and it turns out I did it.

I haven't quite hit publish.

Actually, I might do it before this episode goes live.

I think I will, but I did it.

I extracted a standalone form API.

So all of these ideas we talked about are no longer coupled to Elm Pages.

And it turns out it worked out pretty well.

So I'm guessing that simplifies Elm Pages API quite a lot.

It removed several modules from the API doc sidebar, which was nice.

It's already quite large.

Well, I guess you still have to learn them, but they're just in a different package.

You have to learn them, but it provides an incremental migration path, which is nice

if you wanted to change over from a different framework or vanilla Elm over to Elm Pages.

And it also gives you the ability to create little standalone shareable LE snippets because

you don't need a framework to use this form API.

So yeah, they turned out really nicely.

So what did you end up doing to make this work, to extract this to a separate package?

How did that impact Elm Pages API?

And is your form package influenced in any way with how it will interact with Elm Pages?

Yeah, you know, it's interesting.

At first, I wasn't sure whether some of these ideas would translate to a standalone tool.

But after I've gone through this process, I've gotten a little more clarity on it.

And I think they do translate.

But really, the core idea is this opinion that form data should be managed for you.

And you should defer parsing it into higher level data or errors to a later stage.

Because when people are, you know, designing things that prematurely parse, you end up

with structured data or like semi structured data in your model.

And you what I realized is that by doing that you're coupling to your specific app, because

you're coupling to those data types like your, this is one of the really interesting things

in in Elm is it kind of makes some of these types of coupling more clear, because you

have to be so explicit, you know, not just in what, what state you depend on and things

like that, of course, but also what types you depend on.

Because you can't just say, Oh, I'll take any type, and I'll check if this type is there.

And I'll do these special things with these types.

You know, you can have like, you can have a type variable, or you can have a concrete

type, but you you can't cast things and you can't check what the type is.

And so, so what I what I realized is that when, you know, it's a type of coupling, if

you have like a form field, and you have a message for changing that form field, you

know, for example, check in date changed, if you have a message for that, and then you

have some maybe date, because it parses into a maybe date, you've coupled that to this

type in your domain.

The reason is a problem is because by coupling to that, you don't have a way to abstract

things away from that.

So like, you know, when you couple to something, you have to deal with it as a special case.

But if you decouple from data, and you just say we have low level data, like all we have

is key value pairs, then you just store all of that as a single entity.

So this is just form data.

It's totally unstructured, it's totally untyped, it's just strings, which at the end of the

day, that is like, you know, even if you have a number input field, you can be in a state

where it's invalid, you can type 1.2.

And when you do one dot, that is a string, which is not a valid number.

So you need to be able to represent strings at the end of the day, they're all strings.

To kind of summarize the previous episode, like Elm Form is a lot about handling things

at the very low level, like keeping things in a low level data state, just pairs of strings

So for each field, you will have a value, and the name of the field, and a list of those

is just the form data.

So that's what you use as the low level data.

And you will never transform it to anything else until you actually try to parse it, which

you do when you submit it, basically.

When you try to submit it.

Yes, and also when fields change, when there's a form field event that changes a field, then

you present real time client-side validation errors.

So you're basically kind of putting these two pieces together, but they're completely

So you've got low level form data, and you've got a parser.

And those two things, the low level form data doesn't know or care about the parser.

So it's completely decoupled from it.

It is a completely agnostic data format.

That's low level, it's not coupled to domain types.

The parser does know about your domain types, it parses into whatever types you want to

But by separating those two pieces, the form API can completely take charge of managing

all of your form state, including not just the values for your form fields, but the field,

the term I'm using for it is field status, but like whether something has been blurred

So all of that low level form state is completely managed by the API.

So the end result is you, so the wiring is you have to use form.init in your model to

init it, you have to have a form.model in your model, you have to have a form.message

in your message, and you need to have an update clause that delegates to form.update.

And then you need to render in your view.

And now that might sound like, okay, that's just like what we're used to doing with Elm

Yeah, that's basically the Elm architecture.

Like to call it, right?

The triplets, the Elm triplets.

But this is not the pattern that we're used to using for form packages.

The pattern that we're used to for form packages is wiring in messages and model fields for

every single field.

So this simplifies that.

So it is a single form.model value for all of the forms you might have on a given page.

So if you have multiple forms that you could submit on a page or even throughout your pages.

So this is one of the really cool things.

You know, if you have, you can wire up this form state, you know, in your shared model

between pages and maintain it in a single place.

So that was one of the reasons why I wasn't sure if this pattern would translate from

Elm pages into a more general use case was because Elm pages had this built in assumption

where it had, so it managed the model for the user, right?

So Elm pages as a framework can have its own model and then the user's model sits within

its model, right?

And so it can manage the form state, which is the state for any form.

And then it can have its own message.

So the Elm pages framework can have its own message, which knows about these form events.

So all you have to do is render a form and it renders with this pages message type, you

know, HTML pages message.

And then the framework knows how to deal with that message.

So you don't, that means that you can have a form with real time client side validations

that dynamically manages all the form state without needing to call form.init, form.update,

have a message for it.

All you do is render your form and the framework just takes care of it.

So it's, it almost feels like magic when you're using JavaScript and you just wire up a component

and it just knows how to use this global state.

But it's actually not magic.

And if you look at the time traveling debugger, you can see all these explicit messages that

are being received.

You can still use the time traveling debugger.

It's not magic.

It's just a baked in opinion.

So the magic is that it's decoupled from your type because it doesn't need to know what

type your form parses into to take ownership over that form state because it knows what

form state looks like.

It's just key value strings and blurred, changed, et cetera.

So it feels magic because you don't have to wire it in yourself.

And the reason why you don't have to wire it yourself is because the types are known

because there are those low level data, right?

So it's a list of string tuples.

Whereas if my form was something that I made myself with the custom types and custom updates,

custom messages and so on, then I would have to wire it in somehow.

In this case, I just have to tell you where it is and that's it.

Basically right.

In this case, in an Elm pages v3 application still in beta, you say, you know, you say

render HTML for the form and you give it a form ID.

And that form ID is the unique key that it's going to manage in the dictionary of the form

state for all forms in the app.

I'm guessing you have an ID so that two forms with the same fields don't conflict.

Like if you have a first name in two different pages, they're not using the same data under

So that, you know, that is a constraint of how this package operates is it does operate

under the assumption that you're using, you're giving unique form IDs.

Actually, I think this would be a really good review rule to check that you have unique

I was even thinking like, so you can check one, one cool way to check this.

It would be kind of interesting to check that like you could by convention check that the

declaration name of a form matches the string that you use for the unique identifier.

But you still need the ability to make something unique if you render like a delete button

for every item or or a quantity, you know, quantity field for every item in the cart.

Those each need to have a unique form ID.

So in that case, but it would be possible to just scan the page with Elm review and

say this needs to have a unique form ID.

And if you do, for example, list dot map for each item in the cart, you should use something

from that list dot map in your unique ID to to ensure that it's unique.

Sounds like react where they tell you, hey, don't use the index as the key for this in

I'm slightly confused here is the form ID for the field or for the form.

I thought it was for the form.

So that you differentiate between two forms on different pages or different sections of

The the form ID is for the form.

And then you also have to give a name for each field that's unique within that form.

Okay, so you would like to have something that checks that the form IDs are unique,

and that for each form, the names are unique.

And that would be a best practice just in general for accessibility to have it.

Yeah, for accessibility purposes to have a name for each form field is a best practice.

In the case of the form, it actually doesn't matter if you have a unique ID if you're only

rendering one form on the page.

But that's just one requirement that this API has.

I think it's a fairly reasonable one.

But it would be nice to have either the framework or Elmer view check for whenever you misuse

Yeah, and that would be very nice.

And it's a bit hard to tell which one is more suited because each one will have its own

pitfalls and things that it can check and things that it can't check.

So a bit hard, but yeah, definitely that would be useful.

Yeah, so originally, I wasn't sure if that same pattern would translate outside of Elm

pages or if it would even be interesting because this whole core value proposition was manage

low level state so that you can simplify the wiring.

So you have this magic trick that it's still regular Elm code, but you can just say, render

form, give it a unique ID.

And now you have real time client side validations for this form parser you defined.

And it's like giving you the errors on the fly and you haven't added anything to your

model or, you know, all you did is render something in your view.

It turns out, I think it's still pretty useful.

One of the reasons I think it's useful is because I extracted it out in such a way that

you can do that same pattern in your own application if you want to.

So you know, I mean, other frameworks could integrate this in the same sort of pattern

that Elm pages does.

But in your own code base, you could have a single sort of shared model between pages

that has all form state and you could have like a shared message.

I'm not sure if people use this pattern very commonly.

I'm not sure I had seen it before I kind of came up with this for Elm pages.

Have you seen this before Jeroen?

Of having a sort of framework message and wrapping the page specific messages in the

sort of app message?

I don't know if I've seen it in packages.

I definitely use it at work on something Elm SPA like.

I was going to wonder like, do I do that for Elm Review?

But no, Elm Review doesn't use messages at all.

But at least not in the API.

So no, I do use this in a work project, but I don't know if I've seen it in packages.

So then, for example, in your code base or other code bases that use that same pattern,

you can get the same ergonomics and the same sort of magic trick that Elm pages does where

you just create an app level message and each page doesn't have to worry about wiring up

that form state.

And I have to say, I used to find it to be a bit of a nightmare working with forms in

I used to kind of dread wiring up every single message every time.

And this is a big relief.

I think, correct me if I'm wrong, but the reason why it works so well for you is because

you're using code generation to wire that up partially.

So that's what I'm doing at work as well.

And that only works if you know how something will look right.

You can only code generate things that you're aware of.

So as you said, if something is opinionated or there is some convention around the practice,

then you can make nice tools around it or automate boilerplate writing.

So yeah, doing something conventional or standard or really helps with having a nice experience.

But I think that's partially the reason why things like Ruby took off because they had

so many things around convention.

You correct me if I'm right, because I don't know Ruby at all.

But since everyone did things in a specific way, people can make tools that assumed or

presumed that things were done that way.

And then you can make very nice tools.

That's also kind of what we do with Elm in a lot of ways.

We know that none of our functions will have mutations and all those kinds of things.

And we play with that.

The only limitation is that we still need to resolve the types in a way that makes everything

But apart from that, we kind of do the same with our tooling.

And that's why we have a lot of awesome tools.

The thing with Ruby is that it has a lot of capacity for metaprogramming and patching

And I think that is one of the things that made it successful.

I mean, Rails was sort of built on a lot of those features, being able to use method missing

and just patch in methods that made things ergonomic and gave you conventions for being

very productive.

And I think that's been sort of like a challenge in Elm is like when we have to be so explicit

and this programming language is not, it's the opposite of dynamic.

It's very static, which means you often have to be very explicit, which sometimes means

And so the one downside with having one opinionated way of doing things that you have to abide

by is that you're limited in how you do things.

So if you do that with your form package, if you have an opinion in a way of managing

that form, then you better hope that it's good.

Which you seem to think it is.

I haven't played with it yet.

We'll see later on.

Well, you, yeah, you could share your thoughts once you try it out.

And we kind of talked about this in our original episode where we talked about exploring this

initial API design.

But when you have, so choosing what you have opinions on and choosing what you couple to

choosing what assumptions to make, right.

If you, so this form API has an opinion that form data is a thing.

Form fields are a thing.

They are input or text area elements and they have key value pairs.

That I mean, that is like, that's not my opinion.

That's the browser's opinion.

So I think that that's a pretty solid foundation for an API to build on.

Because you're working with conventions defined by WWC, I think.

World Wide Web Consortium.

You're conforming to MDN.

So by doing that, this has been something that I've been really thinking about a lot

recently is sort of how do we fill that gap where like exactly this sort of sweet spot

that Rails and Ruby were able to fit into where you get to use this magic, which makes

things very sleek and you can give an amazing demo because you're just like, hey, I just

put this thing here and all these things happen.

You don't have to like, okay, I wire this in here.

And I have to have a message that responds to this thing.

And oh, the types aren't right for this.

They didn't line up here.

So we have to change that.

It's like, that's not very sexy.

But you can get a lot closer to that magic when you bake in these assumptions.

The one thing that I've quite liked with that I've very much liked with all the element

implementations where they do code generation and something quite like magic in a way.

I'm mostly thinking of Elm SPA where they generate some of the files to do the boilerplate

between the main application and all the different pages.

The nice thing with that is that you can look at the boilerplate and you can see how things

And in Elm SPA's case, you can even overwrite how the boilerplate writing is done, wiring

I'm guessing with Elm Pages you won't see that.

Elm Pages has a specific, I mean, it tries to, you know, give you the right extension

points to customize things, but it is like, it generates its main.elm, which does its

thing and in that particular part.

So it's more like Elm SPA gives you a sort of customizable main.elm, I believe, and Elm

Pages gives you points to customize that it pulls into the generated main.elm, but main.elm

is not customizable.

And there are some limitations, but there are some conveniences with that.

And it doesn't have to be important to look at how things are implemented under the hood.

It really depends on how much do you need it to understand the whole thing.

Like for instance, Elm Review has the same mechanism where you give it a list of rules

that you config and Elm Review pulls that in and we're good.

No one needs to understand how things are wired in into main.

They just need to understand what a rule is, what it does, and that's it.

I mean, that's the power of a good abstraction.

Of course, if it's possible to abstract out a certain detail completely, then that's ideal.

Now I'm curious, will people be able to see the Elm files that you've generated in the

Elm stuff folder?

If people are really curious on how things work under the hood?

You can see all of the code that's generated.

It's there on the file system.

I wonder if the editor picks it up, like all the connections.

Oh, this function is used in this generated file.

Because that would help with understanding.

That could get people lost maybe in some cases, but maybe, I don't know.

That's a good question.

It might not, because it does create a separate folder that copies over the Elm.json.

I've definitely thought about these kinds of considerations.

It gets interesting creating an intuitive experience where the editor gives you what

you're expecting there.

I actually think that it won't, because I don't think it does that for Elm Review either.

You can't see where the config that you've defined is used.

So we talked about decoupling from the domain-specific data types for having a separate form parser

and a separate sort of low-level data type.

We haven't yet touched on using that low-level data type for submissions.

And this is actually another one of the key pieces that Elm Pages uses for this magic

is if you're able to do code sharing, then you can send that low-level data.

So what typically will happen with a front-end application these days is you'll have some

You manage the form inputs programmatically through JavaScript somehow.

And on submit, you intercept the on submit, and then you end up encoding some JSON objects

that you send up to an API or something.

And you can do that using my Elm Form API.

I have a with on submit, and you can use that to...

You can even write your sort of form parser in such a way that you can take your form

field inputs, and you could even parse it into a JSON object if you wanted to, for example.

You could parse it into a tuple with a JSON object and a nice Elm type, which you can

use to show optimistic UI, to show the in-progress creating item that you've got.

There are all sorts of things you could do.

It's quite flexible in that regard, but that works.

But that's an extra layer of glue.

You know, Lamdara talks a lot about...

You know, Mario has given some great talks about this philosophy in Lamdara of removing

And I think he's done a brilliant job with that.

And I believe that this actually removes an additional layer of glue that Lamdara still

has in it without using this approach.

Because if you think about it, it's a lot of glue to turn these...

You know, to do all this managing of form input data and sort of parsing it at various

It turns into a lot of glue.

And then when you want to send that data, that you're turning this sort of unstructured

form data into structured data that you can send up to the server, even with Lamdara,

when you can say send to backend, you still...

There's an additional challenge that you can't trust that user input because...

Because someone can just craft that same request to the backend.

So they could take the underlying bytes that are sent, they could hack on that, and they

could say, okay, well, you know, this represents an Elm type, which is a valid username, which

valid usernames must be this many characters and can't have an at or whatever.

So that's a nice opaque type, right?

It's impossible for that type to be created unless it goes through that code path, right?

Except with Lamdara wire, when you do send to backend, it just has a bytes decoder.

And magically, that opaque type comes into existence from those bytes.

And if you were to hack those bytes and give it a string that's not a valid username, so

you actually can't trust it.

And, you know, if you look at the meetdown code, Martin Stewart pointed me to some code

in that code base, which is a Lamdara app, really, really cool service.

And he has a few helpers that sort of have security around that.

So he's actually considered that, you know, you know how much we love opaque types and

getting guarantees around that, but then you sort of lose those guarantees and you have

to revalidate all of your opaque type constraints.

And again, only at the edges, but still you have to do it.

It starts to feel quite messy.

And then you're like, wait a minute, can I really rely on this?

And now you're dealing with opaque types that you can't trust.

The whole point is guarantees.

So that's like, that's the stuff of our nightmares.

I mean, if you can't trust opaque types, then who can you trust?

Think of the children.

So there's the problem.

How might this form API help with that?

Well, I'm glad you asked.

Hey, Dillon, how can your form API, how can it help with that?

Excellent question, Jeroen.

The answer is you defer parsing that data until you receive it on the server.

So now you, with LambdaRacend to backend, what are you sending?

You're sending low level key value pairs of strings.

Which is the underlying, I mean, this is, as far as the browser is concerned, that's

the only thing that exists.

That's what form data is.

It's low value, low level key value pairs keyed off of the field names.

So you can submit those.

You don't even need to write an encoder or anything for the submission.

You just let it submit those.

You can use send to backend to send that low level data.

Now in your LambdaRacend backend, when you receive the low level data, now you run your

If you want to, you can run that.

I mean, you run that parser on the front end.

You know, you render with that sort of parsing logic.

So you get your client side validations in real time as you type.

But then it reruns those same validations because you're using the exact same parser

on the backend.

Now that's your source of truth.

So now the source of truth doesn't come from data that's serialized from the front end

to the backend.

You've got low level data as the thing you're sending to the backend.

And your source of truth is your code sharing of your form between the front end and the

So now you can trust your opaque types because you're running them in a trusted environment

in the backend.

And I actually did a small example with a really very simple little LambdaRacend toy

app I have, but I tried it out and it works great.

You just run your form parser on the backend and get your data and check if it's valid

or invalid and it works beautifully.

So people can still hack the bytes that are sent, but the backend will just refuse it

because, hey, this is not a opaque type.

And I'm guessing they will have to handle that somehow.

But that's not a lot of work.

Do you mean the opaque type of the, what they're sending is key value strings?

What I mean is they get the backend gets the low level data and they parse it into an opaque

type or an error, a result, a failing result.

And then they need to handle the successful case and the error case.

And that's running through backend code.

So that can be trusted.

So the thing is like conceptually, any user, trusted or not, has the ability to send data

And it's the server's job to then take that unverified, untrusted data and decide whether

to trust it and to do something with it or ignore it.

And it should probably never really trust it anyway.

So by deferring, parsing it from low level to structured data until it gets to the server,

you are following that conceptual mental model of not trusting it because you're just like,

hey, you can send me key value pairs, right?

Like you can go to, you know, I could open up some, you know, web tools and make a submission

to Amazon, to some Amazon forum and type in the key value pairs I want to.

And I could try to like put some unwanted values in there, but you know, and they can

decide what to do with that in return.

But it's the job of the server to decide what to trust because the client can't be trusted.

The client can be anything.

And we don't really care what the client is.

Like the client could be Vim or it could be a script or it could be a curl command.

And we don't really know, like we can kind of try to put some things through JavaScript

onto the page to put like a, you know, a little hidden field to try to prevent that.

But we can't really know that we can trust the client because the client can do anything.

But the server we control.

It's kind of weird that we have both the sentence, the client can't be trusted and the client

But try to merge those together.

I don't know if I've heard that one.

I've heard that one before.

You haven't heard that one?

It's mostly for restaurants or.

I've heard the customer is king.

Customer is always right.

Oh, maybe that's just a.

It's French because customer is client.

That makes sense.

That makes sense.

So that's a French English joke.

It's always a bit hard to figure out who can you tell those jokes that mix two languages.

I have some jokes that I can only tell my family because they're like French and Dutch.

And it's a mix of those.

There's some good ones, but you can only tell them to specific groups of people.

And it's such a shame.

We can tell the Elm React jokes here, but that's about it.

So because we're dealing with low level data, you don't have a lot of guarantees around

it to start with.

So for instance, how do you make sure that in a form that I'm going to send the backend

that I have not forgotten a field at the backend expects?

Like you know, usually we try to create a nice opaque type and that opaque type in it,

it expects something of a specific shape.

Like if it's Jason, then you parse it and you expect it to be a specific shape.

If it does, then you get an okay.

And otherwise you get an error.

And that is always a bit tricky to get right in the sense that if you forget it, then you're

likely not going to notice it unless you write specific tests for that, which I personally

don't, but I probably should.

So how do you avoid that problem with your form API?

Now do you mean like a required field?

Yeah, let's say there's a required field that is the first name and you forgot to add a

field for that because you know, you and me, we both like guarantees and all those things.

And we like things type check.

And I would like to check that this first name is never forgotten.

So how do I ensure that?

Is there something in your form or is it just something that you have to write tests for?

So you can, you can write a required form field.

And if that field is absent, then that validation will fail.

So your form will just be parsed as invalid and you'll get a client side, you know, you'll

get a client side validation.

You can forget to render a form field.

The form fields we discussed in the last episode on this, that it uses this sort of Elm codec

like pattern where there's a pipeline where you sort of pipe through adding each field

in the form, and then you get those in a Lambda and you, you use that to parse into the data

type and you use that to render your view with all the form fields.

You could forget to render a form field.

And if it's, you know, if it's optional and you don't do any validation on it, then that

that is something that will pass that validation because it's just like, yeah, there's no,

this field isn't there and that would be valid.

So that is something to look out for.

Again, an Elm review rule would be a great, great way to check that you've used every

field in your, in your view.

I think the unused parameters rule would mostly work, but you probably should not.

It could be used still because you need to use it in two places.

Oh, cause you use, you use your form fields in your, in your parser definition.

You use them in two places.

You use them in your view to render a custom, custom view, however you want to render it.

And you also use it to do dependent validations, to combine together all of your fields into

So in practice, would you write tests for that?

Yeah, I think, I think it's a great thing to write tests for.

Yeah, absolutely.

That would be a really good thing for using Elm program test for.

Yeah, that would actually work quite well.

So we talked about sending this low level form data with Lambdaera and I was pretty

excited with how nicely this worked.

And again, if you wanted to, you could even take it a step further and have these app

messages where the form messages don't need to be wired in for every single page, right?

You could have a single application wide message for your form messages and you just render

your, so you could get that same magic that Elm pages sort of bakes in.

You could bake that into your own framework or there could be a Lambdaera specific framework

that bakes that in.

So there's all sorts of opportunities for sort of getting that magic there, which is,

which is pretty cool.

But yeah, I was pretty excited with how this, this pattern still works with Lambdaera.

And to me, this, this takes this like Lambdaera philosophy of like removing glue and removes

a big piece, right?

Cause you're like, wow, I can build an app with no glue.

That's amazing.

So you're saying I can like send data to the backend with no glue and receive it from the

backend with no glue and it's all real time data.

That's amazing.

It's like, yeah, no, no glue.

Well, what if I want to like send up some data, like input some data and send it up?

It's like, okay, you write a message and you, it's like, oh, okay.

That's, that sounds like a lot of, a lot of glue.

So I feel like, I feel that this removes a last remaining piece of Lambdaera glue if

you, if you use this pattern in Lambdaera.

You're removing glue while keeping things working.

In a way that is predictable.

Now this gets to an interesting question in the case of Lambdaera.

So, you know, Lambdaera also has this really interesting guarantee that you can migrate

data and it's like a guaranteed safe migration because you have to account for all of your

data and how you, even how you would deal with messages that are coming into the server

from an old version.

So when you say migration or migrating messages and models, it's when a old client talks to

a new backend or the other way around or in all those directions that are possible, it

migrates things in a way that they can communicate safely.

And you can migrate your front end model and to new versions of the app and all this.

So that's one of the big value propositions of Lambdaera.

So then you take this form data, which maybe let's say before it was more structured form

data, you had a check-in date was in your front end model and your checkout date and

your first name and last name.

And now, and you had a send to backend, which took that and you had a specific message for

And now you lose the migration of all of that type safe data that knew exactly what the

shape is and what you had in the front end and all of that.

So this is something I've been thinking about a little bit.

I think that let's say that you, let's say that you add a new field and it's a required

Now you are going to, if the user's on that page, now suddenly there's a new field, but

there's also going to be a client side validation that says you're missing a required field.

Wait, just so we're clear, the client is on the old version or the new version with a

Well, if the client migrates to the new version.

I hope I'm understanding how it works correctly.

Maybe it does it not migrate.

I think it migrates.

I could be wrong on that.

I have to check, but whatever it does, the source of truth is the form parser and the

backend checks that source of truth and the backend is able to give validation errors

So, I think it lines up.

I might be missing a detail about how this works, but I believe it's the paradigm still

works out for at least for cases where you're adding a field.

If you're removing a field, of course, it's all kind of messy.

There's no simple, neat way to migrate a form as somebody is entering data into a form.

I think with the Lemdara, if you have a frontend in V1 and it sends a message to your backend

in V2, then that message will be, that sent to backend message will be migrated.

Because your data is low level, you will not have a migration.

Although I'm guessing you could write a migration, even when the types don't change.

If you wanted to create an introduce a new variance name, just to be explicit about that.

Yeah, you absolutely could.

That's a good point.

If you wanted to, you still could.

But in this case, you would probably say, hey, you're missing this required field.

Just refresh the page because you can't migrate.

You can't add missing data to form, right?

And I believe it, looking at the Lemdara docs, I think it does migrate the frontend model

So you're going to get an update where it's showing you the new form fields.

Then it's mostly an issue about when do you get the update and when is the, how does the

deployment work?

But at the end of the day, I think it's still like, so my bottom line is I feel like you

don't lose a whole lot by using the lower level form data to send it to the backend

because at the end of the day, you're going to need to handle the possibility of an invalid

form being sent to the backend.

There's no avoiding that because clients can send anything.

Even if it's only supposed to send through Lemdara, you can still hack it and send bytes

It's like, that's just how servers work.

You can send data to them.

And so I think that this approach works really well where you're just saying the source of

truth, the gatekeeper is the form parser and you can code share that to show the client

side validations and turn it into structured data on the backend or get validations, which

we can send back to the client and say, Hey, you must have bypassed something, but you

gave me something invalid, right?

Or you can also do backend specific validations.

That's also possible with this design.

But I think it's a sound approach because at the end of the day, you still need to handle

the possibility of receiving invalid data on the backend.

And if there's a migration and suddenly the front end just migrates to something where

now there's a missing required field, you send it to the backend.

It doesn't matter that you didn't do a migration.

The new version of the code will handle that validation and then you'll also see that validation

error on the front end.

So I think it's a sound approach.

Maybe some Lambdaero nerd will give me a corner case.

I would be very interested to hear if there's a corner case where it really isn't as safe,

but I think it works pretty well.

So as far as I can tell, this works pretty well when the backend expects the same low

level data that you're using in the front end.

So that works very well on pages.

I can work very well in Lambdaero as well.

How does that work with REST APIs or GraphQL?

Do you just keep that low level data in the front end?

And then when you try to submit, you need to parse that and make the REST or GraphQL

request, but you don't have that same technique that you use on the backend about getting

low level data.

Yeah, that's a great question.

So and at first when I was extracting this to a standalone API, I wasn't sure if that

would pan out and if the pattern would apply well to a use case where it's front end only

Elm because we're talking about full stack on pages v3 and Lambdaero Elm.

But actually, I think it does work quite nicely.

And one of the patterns I hinted at earlier is that so in your form definition, you take

all of your form fields and combine them and you can add additional validations and dependent

validations between the form fields.

And then you tell it how to parse that successfully or unsuccessfully into structured data.

So you could parse it into a record just like a JSON decoder.

You can parse into a record, but you could parse it into any data.

And as you're parsing it, you're basically telling it, you know, so the most common pattern

will have will be to have like a type alias for a record and to use that record constructor,

you know, just like in a decoder, you say decode dot succeed user, and then you just

keep adding on these decoders to applicatively add data to that constructor.

But that record constructor is just a function.

So you could just as well say a function that takes this value and this value and turns

it into these values.

And you can put it into a record if you want.

But what if instead of putting it into a record, you created a tuple with a JSON encode object

and the structured data if you wanted some structured data to show in a pending UI to

show this is the item that we're creating right now.

But you can also construct, you know, you could even construct like an API request payload

thing that maybe there's a custom type for which endpoint you're sending to and maybe

there's a JSON encode value for the payload to send.

Or you could build up, you know, an Elm GraphQL input object or whatever it is.

So like, because you have the data right there, you have all of the fields directly to combine

into whatever data type you want.

So, so when you render the form, you can say, with on submit, and, and then you can either

get the valid or invalid data.

And with the valid data, you have the successfully parsed data type.

So we've talked about low level data with forms.

Do you see any other places where we could use this technique?

I'm thinking maybe HTML bytes or I mean, we use it for JSON to some extent, maybe?

No, it's not the same thing, right?

This is a very good question.

I'm trying to think now.

I have noticed as a general principle, with API design, sometimes when I have a type variable

in an API I'm designing, I'll try to sort of squeeze out the type variable by distilling

things down to lower level data.

So that's sort of like something I look for when I'm designing APIs.

But but yeah, that's a good question.

Like what are the pain points from because I really feel like, you know, I might sound

like a broken record now, but it just feels like the way we were working with forms in

Elm before was so tedious.

And we worked with forms that way for so long.

And it like it wasn't it wasn't good.

I mean, we were clueless and now we are glueless.

Let's just say it's been a formative experience.

We needed a reform.

We can all agree.

I almost feel like I have just made one pun and you're like, okay, now I can throw them

Never never look a pun artist directly in the eye.

Your kids are never going to look you in the eye.

It's too dangerous.

It's a great question.

Can you think of anything that might benefit from low level data?

I don't have anything in mind.

But I'm sure the listener will tell us if they come up with something.

Actually, there's one use case.

Simon Liddell, he made a Elm URL package.

I don't remember the exact name.

And that is basically the URL as a string.

Let's keep it as a string.

Let's pattern match on it as a string.

Or as a list of strings, but still pretty low level.

So yeah, I think that works instead of having a parser with a type and yeah, URL.

Yeah, actually, you know, this technique of defunctionalization too, in a way is a form

So defunctionalization being instead of having like closures of things to execute, you turn

it into data, just like a message is defunctionalization.

Instead of having like an on click function, you have a message.

And each and the handling for each message will do a different thing.

And so I think there are certain ways to kind of combine this idea of defunctionalization

and decoupling to lower level data.

For example, an HTTP call.

You could so like Mario and I were discussing with the Elm pages back end task API, it doesn't

use Elm HTTP, it has its own back end task HTTP API.

So how could you take a request, but the request is this sort of opaque thing that you can't

do anything with.

So if you extracted that out to its parts and had the request object be just a data

type, maybe that would make it more shareable.

Yeah, because that would be a common denominator.

Yeah, I'm not sure it's exactly the same principle, though, because like, there's still, it still

knows about some of the types in your specific use case, whereas we're talking about not

knowing about anything like form data doesn't know about anything.

Yeah, I'm gonna have to think on this a little bit.

But but yeah, listeners, please do tweet at us or message us in the Elm Radio Slack channel

if you think of anything.

Don't talk about it too hard, because I don't want to delay the publishing of this package.

So there is one thing I feel I should mention, which this package works with Elm CSS.

And it works with Elm HTML.

It does not work with Elm UI, unfortunately.

The reason is because this package has strong opinions about form fields.

And Elm UI doesn't expose a way to natively render form fields, except by dropping into

But there's no way to render semantic form fields.

So like, this library, this package assumes form field events, like native HTML form field

And it also just doubles down on trying to use forms for accessibility and wrap things

in form elements and that sort of thing.

So that's just sort of a baked in opinion.

I would love to see Elm UI expose some way to render form components, but at the moment,

it doesn't expose that.

So it was quite an interesting experience, sort of.

When I was building this, I first started with an experiment to extract it to a standalone

I just took this Elm package starter repo that I have, and I just copied the relevant

code in there, and then without having the specific Elm pages dependencies, and then

I just saw what was read.

It was really, really interesting.

Like a few of those things, like those were the points that I needed to abstract away.

And what I ended up doing was really like doing an inversion of control in a lot of

those places, which turned out to be, I think, a really just a nice way to design it overall.

For example, like the original API just sort of reached in and grabbed state from the sort

of Elm application state, which it knew about.

But then when you decouple it and it doesn't know what Elm application state is, now you

have to pass that in.

So you say, is it submitting this form?

And then what I ended up with is I ended up creating like a little adapter module.

It's just like using the adapter pattern.

So it's a very thin layer that, so instead of calling form.renderHTML in an Elm pages

app, you can call pages.form.renderHTML.

And whereas the standalone package takes a record where you give it submitting equals

and then however you want to manage that, you can say model.submitting in Elm pages,

it knows whether it's submitting because it handles submitting for you.

So when you say submit, Elm pages takes the low level form data.

It has, it keeps track of the state in its own framework level model of the in progress

form submissions.

And it knows how, given its model, how to say whether a field is being submitted.

That was, so it was pretty cool.

Like I basically just said like, oh, all these things that depend on the Elm pages application

I'll invert that.

Submitting equals some Boolean.

And then in the adapter, I took all that code that had the logic that knew about the Elm

pages application state to deduce whether it was submitting or not.

And just gave that as the value for submitting equals all that logic I extracted that knew

about Elm pages.

So it's pretty cool.

It was a lot cleaner than I expected.

When you started, did you think it would not be possible?

I also like had a few places where it was coupled to the backend task API, because you

could do backend specific validations.

If you wanted to, for example, check that a username is unique or try sending an email

to check if an email is valid or whatever on the backend.

But it turned out I was able to extract that out into this API where you combine together

a set of form definitions, and it will try running the parser on any of those.

And it turned out it was a very light wrapper to be able to parse into a form.

Basically what it turned into is, so instead of the form parsing into a user record, or

the form parsing into a JSON encode value, what it ended up being is just the form can

parse into a backend task.

If it fails to parse into a backend task, that means that the client-side validation

If it succeeds in parsing into a backend task, now you can continue doing backend validations.

And I ended up splitting off this thing that you can render to kind of give server state.

So you can give error messages from the backend if you want to.

I was actually not expecting it to work out.

Maybe not even at all.

So it was a pleasant surprise.

But now that I did it that way, I think it's really nice also to be able to just look at

the docs separately, to learn this tool separately, for people to be able to talk about this package,

not just like, oh, it's this Elm Pages specific thing.

It's the same API, if somebody's using Elm Pages or somebody's using Lambda or somebody's

using Elm SBA, they can all talk about the same package.

I think it might add a bit of confusion if you're used to neither, because then, oh,

what is this form?

Oh, well, this is in that other package.

Oh, I didn't see that one.

But apart from that, sounds like a clear win to me.

Especially if people start using it in other places, and then they start using Elm Pages,

then the learning curve is a lot smaller.

So that's nice.

One of the tricky things, design decisions that I dealt with here was, there's this config.

So I have this options type, which when you render a form, you give it options, which

has like, it must have the form ID, the unique ID for the form.

But it can also have a form method.

So a form can be get or post.

This is like, most front-end only applications aren't going to care about this, because they're

not progressively enhancing a form, whereas Elm Pages uses progressive enhancement to

So I have Elm Pages examples where I use get form submissions, because when you do a get

form submission, it appends query parameters to your URL and reloads the page.

So for example, if you're doing a search query, that can be a really nice way to do that.

And you can progressively enhance that.

So instead of doing a hard page load, you can do an XHR request.

But so anyway, the Elm Form API has this baked in opinion that a form method is a thing that

And most people will probably ignore it, but it does have that baked in, even though probably

Elm Pages apps will be the only ones to use it.

But a Lambdaera app could use it too, if it wanted to, or a front-end only app.

So I left it in there because it seemed like a reasonable opinion that that exists, since

it does exist in the browser.

But yeah, so I created this options record and use a little builder pattern to build

up these options.

And I share that options record between the Elm Pages package and the standalone package.

So I'm guessing that means that whenever you need to do a change in the form package, you

will also need to update that in Elm Pages.

So you will need to bump them together to keep them in sync.

What I ended up doing is I just basically in that little facade module that I have in

the Elm Pages API, pages.form, what I did is one of the arguments is the standalone

packages options.

And you transform that to the options of Elm Pages.

Right, exactly.

So I supplemented.

So the Elm Pages options are a superset of that.

So I said, you know what, I'm just going to, so that they can share as much as possible,

I'm going to share this options type between the two.

Even though there are a few additional options that you can pass into the Elm Pages one.

So I think that worked out pretty nicely, but that was definitely a subtle design decision

So if people want to know more about Elm Form and Elm Pages, or your form package, did you

even call it Elm Form?

What is the name?

Did you announce it during this whole episode?

No, I somewhat was leaving room for the possibility that I changed my mind about what to call

But yeah, I think for now I'm calling it Elm Form.

I think Elm Form, I don't think I need to get too clever about it.

It does kind of like represent a, I think, pretty different philosophy around how to

deal with forms in Elm.

So I definitely have considered like, does that deserve a different name?

But yeah, I'm just calling it Elm Form.

Not Elm Awesome Form Meatballs or something.

That sounds delicious.

It does sound delicious.

Yeah so Elm Form.

So you know, we'll link to the package docs.

Hopefully by the time this is live, we'll link to live package docs, not the preview

And I'll try to get some nice ELE examples there, because that's one of the cool things

we can get ELE examples.

But there is a nice examples folder in the repo, which I'll link to.

And yeah, I'll also link to a few examples of it in action in Lambdaera and in Elm Pages.

So you can sort of compare the Elm Pages one.

It can get pretty sophisticated because you can do all these use cases like dealing with

in-flight submissions and parsing that data to get optimistic UI and things like that.

But yeah, so I would start there.

And if you have any feedback, let Dillon know.

And Jeroen, until next time.

Until next time.