Exploring a New Form API Design

Exploring a New Form API Design

We dive into some ideas for a new Form design, pulling in a lot of our favorite Elm techniques to make it safe and intuitive.

PublishedJuly 18, 2022

Episode#61

Parse, Don't Validate
etaque/elm-form
Codecs episode
Some similarities in the design for how you build up Custom Type codecs with elm-codec

Transcript

I have to admit, I'm a little bit intimidated by our topic today.

Me too, a little bit.

I feel like people will have expectations and I'm not sure I'll be up to par, but we'll

Yeah, forms are a formidable topic, if you will.

It really is a complex problem space, I think.

And it is ambitious to...

I mean, working in a type safe world, it does put an extra burden on us to design all the

pieces to fit together neatly, especially when what we're doing is essentially data

intake and data processing and throw type safety into the mix.

And that's a lot of responsibility to make a really nice API for transforming things

in a way that we can parse, don't validate.

And at the same time, all these concerns are working together.

The logic for presenting the view, but that can be dependent on what you've parsed and

how you render individual fields and how you present them.

So it's going to parse into the appropriate thing and how you display errors for each

of those fields, how and when you display errors for those individual fields.

So that's sort of what I hoped we would get into today.

I don't have a particular conclusion about which form people should use, which form API.

Rather, I sort of hope we can explore some of these different design considerations and

really what I'm hoping to do is talk through some of my ideas for how the state of form

APIs could be improved.

That's really what I want to get into.

Yeah, let's do that.

I also don't have any specific package that I want to recommend because the one I'm using

at Humio is custom made and slightly different from the ones that I see on the Elm packages

I'll be curious to hear about that.

Dillon, what is a form?

Oh, it is a form.

Well, I mean, I think, again, it is sort of these two different concerns.

How do you present something to the user that they can input data into and how do you take

that data and turn it into structured data that might have errors, also known as parsing.

Sometimes we call that...

Decoding, data dating, parsing.

And, you know, ideally, I think that parse don't validate is really key in my mind to

an ideal form API because, sure, you can sort of treat everything as strings or low level

data and leave it at that and just tack on errors.

And that simplifies a lot of things, but it doesn't feel right.

It feels like we should be able to parse things into nicely structured data.

At the core, I think it's that and that data, then the purpose of it is usually to send

it to a server.

You sort of front end only forms where we're not really persisting it to a back end or

interacting with some external service.

But usually that's what it's for.

So a form is a series of inputs that you can turn into either a list of errors because

the form is not complete or has problems or to something, let's say, final that you can

submit to a service, a server, another function, something like that.

Just a series of inputs that you want to handle in a nice way.

Yeah, absolutely.

And that you would like to make some nice interactions with, especially with regards

to validation errors and whatnot.

Yeah, absolutely.

You want to give really nice user feedback and you want that feedback to be meaningful,

precise and in sync with what your actual back end expects.

So that's like a little bit out of scope for this discussion, but maybe a bit of foreshadowing

for a future episode about how might you keep your form parsing logic in sync between your

front end and back end?

Well, wouldn't it be handy if we could run the same code on both sides and have that

be Elm code that so we know that they're in sync.

And yes, of course, I'm talking about the new Elm pages release, which we'll definitely

be talking about at some point.

Forthcoming Elm pages release.

It's not released yet.

So it's not new.

You have more work to do Dillon to release it.

So don't give them false hope like it's available today.

No, it's going to be awesome.

I've been thinking about forms a lot because it's pretty core to the goals I have for Elm

pages v3, especially being able to take input, user input and turn it into trusted validated

And you want that feedback.

You want to share that feedback about what makes it invalid to the user.

And you want to be able to use that same logic to get it into a trusted form, you know, trusted

structured data that you can use to do things with, you know, with those assumptions that

you made with your parsing logic.

So if you if you say something is a valid username, then you need to be able to trust

Before we dive into that, I want to know whether you think what a form API is for because when

I was working with React back in the day, it's not like I did it for a long time.

But when people said, hey, I'm looking for a form package, it felt more like people were

looking for how to create form UIs.

Like I want to say here is an input and it expects a number.

And based on this condition, it shows up or doesn't show up.

And it will look like this.

And I feel like the packages that you see in the Elm community, they don't have that

They don't have a visual.

They only have that logic about validating, parsing, showing errors, but they don't have

that particular point of outputting or rendering a UI.

When you say rendering a UI, do you mean like rendering a field that's like a password input

type, so it masks the characters versus a text area?

And with a specific UI, it looks good and you can just put it in your project and use

it and you will be happy.

And it shows that I see.

So I think first of all, starting with what would vanilla form handling look like in Elm

is a very good place to start.

I like that a lot.

For the problem of like presenting the form versus kind of parsing the input, that's a

great distinction.

There definitely are helpers in some of the common packages in the Elm ecosystem that

help with this.

There are a few considerations here.

So the different approaches you could take, you could completely leave it up to the user

and say, all I do is parse data.

I don't have any concern about how you present it.

That's up to the user.

So there are different approaches you could take this.

You could leave it completely up to the user to deal with presenting the forms.

And all you do is you parse the data from those forms.

You could sort of provide some helpers for presenting those forms.

That's kind of the approach that one of the more popular packages out there, Itake Emilian,

I think is his name.

I don't know how to pronounce that properly.

But this package Elm form is like a sort of decoder style API for parsing form input.

And it provides a few helpers for displaying fields.

Like nice looking fields, you mean?

It doesn't have any opinions on how they look.

And I kind of think that that makes sense because at the core, you have essentially

three different elements, maybe four.

You have inputs.

That's kind of the broadest one because a radio button is a type of input.

Text input is an input.

Password input is an input.

A data input is an input.

You have buttons.

Texts can be used to submit forms.

Actually you can create that using an input element.

So that's sort of just a special case.

And I mean, depending on how you're building your forms, if you're not necessarily concerned

about building it as a semantic HTML form, where it's actually like a valid form with

a button that would actually do something without JavaScript, then you could say like,

you know, that's not even a concern of the package.

Here's a message you can attach to it to submit or a function you could use to validate or

whatever and wire that up to whatever element you want.

I don't have an opinion on how you render that.

And then you've got a select element which can have dropdowns and you've got a text area.

So there aren't that many.

And a text area is really kind of like an input text.

It's just like really it seems like it should be a different input type.

There's probably some historical reason for why it's not.

You mean it should be the same or it should be a different one?

Because there are different ones.

I mean it should be like, why not have it be an input element with type equals text

area or something like that.

Like it's just a, at the end of the day, it's functionally equivalent to input type equals

text but it's presented differently but that's the same for input type equals tel for telephone

number or input type equals email.

It's like functionally the same as input type equals text.

You know email is maybe going to run a client side validation automatically unless you have

a no validate attribute which usually is actually what you want to do because you want to present

your own error messages however you want to present them and have more fine grained control

over what error messages to present and how and when to present them.

So at the end of the day, my point is that the concern about how to present it nicely

I personally think is a separate concern from sort of presenting the raw input fields and

parsing that data.

Those two things I think should be a single concern of how you present the low level fields

and how do you parse the data.

And if those two things know about each other there are certain nice things you could do

because a date input has certain implications for what type it will parse into.

Presenting it in terms of nice styles and that sort of thing.

My thought on that is that that should be a separate concern.

Not that you couldn't have nice abstractions to help with that but I think that you can

handle that pretty well by just saying, hey, I know how to render these fields that you

have and hopefully the form API you have knows whether it's a date field or a text field

or a password field and it knows how to put the right attributes for that.

But then it should allow you to put other HTML attributes to style it.

That's how I think of it.

Yeah, same here.

I feel like it could be useful to have the form API provide those helpers but at some

point it's going to conflict with what your UI designer will want or what you will want

the application to look like.

So it can help you get started maybe to get started with having a nice looking thing but

in the end you will want to replace it at some point or have it be quite customizable.

I think I mentioned this before but I feel like that's also the same reason why we don't

have a lot of UI packages with nice inputs, with nice buttons, with all those niceties.

Except the ones that are like this is a standard material UI from Google.

There's a package for that.

And that makes sense because if your company says or your team says we're going to build

it with this material UI style then this makes a lot of sense.

It's going to work for you in the long run or in some version of the long run.

But otherwise it's unlikely that it's going to work out.

So I do feel like they should be separate.

Because you can get backed into a corner where you have a nice getting started experience,

things look really good and you're like wow this is simple.

There are so few decisions I need to make.

It gives me what I need.

And then you get stuck.

And that's no fun.

No, no, you don't get stuck.

You start saying well we can't do this because this library doesn't allow us to.

Or you start doing hacks or you start forking it or you start using a separate library for

those specific inputs.

So I feel pretty strongly personally that the way that the view is presented should

be extremely unopinionated and flexible except for the basic low level fields.

That should be opinionated.

That should be like hey if you're building like if you say here's a password input field

that our API helps you do that.

If you say here's a date field our API helps you do that and it helps you parse that because

it knows things about the expected format that will come from the date pickers that

are going to be presented to users in their browser.

So I feel like we agree that presenting a nice UI is not part of what a form API should

But then what is remaining is our potentially three things in my opinion.

Two of those I totally agree with the third one I'm still not sure.

One of them is handling errors like validation errors and making sure they look good and

that they appear when they should based on the user's interaction.

We can go more about that later.

The second one is doing the actual validation saying this field should take a phone number

and a phone number looks like this for instance.

And that could potentially be a whole separate API in practice and a whole separate package.

And the third one is what to show when.

So like for instance having conditional fields, conditional inputs presenting them how, presenting

them where and with some configuration.

And I feel like that last point is what a lot of form APIs, at least what I've seen

in React land, they get very hairy.

They try to customize everything using like one record to be able to express anything

and everything.

And you were basically inventing a new DSL and at some point you're going to get stuck

because it's not going to be able to do what you want at all or in a nice way.

Is that what you feel as well?

Yeah, I'm definitely on the same page with you here.

And you brought up a lot of good insights here I think.

So you said looking at fields and potentially having that be its own API.

So one of the things that I encountered, so a little bit of background, I tried to sort

of create a form API for the Elm pages v3 project from first principles.

And I built an API and I wasn't sort of looking at docs for other form APIs.

I was just sort of like what do I need here?

And to a certain extent I'm coupling it to certain framework elements that I can hook

into which a vanilla Elm form API is not going to do.

But otherwise I'm trying to solve a lot of similar problems from first principles.

And it's actually a really good exercise because I built an API.

I actually was really happy with certain pieces of it and certain pieces of it I was very

And I basically threw it away and rummaged through the pieces that I liked to salvage

them for a new from scratch API.

And I came up with a totally different approach.

And then what I've done now is I've sort of surveyed a lot of these popular form APIs.

I've asked people what APIs they used.

I've gotten some like code snippets of some homegrown solutions and looked at those, looked

at what's in common, had some discussions with people.

It's been really cool to compare and to see the roadblocks I ran into with my initial

design, the things I tried with my new design, which I'm really pleased with and to compare

So for like one of the things I hit up against with my initial prototype that I was really

unhappy about was I made it too rigid because I was trying to make it this combinator style

pattern where like we've talked about in the past, like with Elm GraphQL, this unlocked

some really cool things to say, hey, is it coming?

What do you mean with a combinatorial API?

Basically the, not combinatorial, but like a combinator.

Just meaning things you combine, but I think of it as like a recursively defined thing.

So like a selection set, so you could either have a selection set and a field in Elm GraphQL,

or you could say a field is just a special case of a selection set.

It's a selection set with one selection.

So it's kind of like for people who don't know GraphQL, JSON that decode value can contain

a JSON decode value again.

So it's like a recursively defined data type and you can sort of deal with it in a very

composable way because of that.

You don't have to go into these special cases where you have to deal with it differently

if it's a field or a selection set, or if it's a JSON object or a JSON string or whatever.

So at first I thought, well, that seems like a really nice approach.

I'll make it really composable.

And I said, why should a field be different than a form?

It's really the same thing.

It parses into something and a field is a special case of a form.

Which seems like a really good idea in theory.

In practice, I didn't like it at all.

And the reason is because it made it too rigid both for presenting the view and for doing

So like, for example, if I wanted to do password confirmation, check that a password matches

a password confirmation field.

Or if I wanted to select a dropdown and based on that show a different field.

Yes, you need to do a lot of and thens or something.

And then and then when you try to build that with something where you're actually like

presenting the view, the types become so rigid that you can't really solve the problem the

way you want to.

And you get stuck.

You get stuck in a corner.

So I basically found that doesn't like it's not a nice way to work with forms.

So what I arrived at for when I threw that design away and built a new one from scratch,

what I arrived at was embracing these two things being different.

And I said, OK, I have fields and I have forms and they are different things.

And I'm essentially going to declare these fields in an applicative style.

So meaning, you know, you pipe things just like a JSON decode pipeline.

You can say required field, optional field, and it's applicatively building up this thing.

So you usually start with like succeed user and then you say required first string decoder,

required last string decoder.

You decode that into a user.

So similar approach.

You build up a form and you you pipe in all of these fields.

So you say with this field, with this field, with this field.

And if all of them pass their validation, because I'm guessing you're also passing the

validation functions, then you have something that it validates.

So that's that's the thing that turned out being really nice about this approach is now

you have a field that has this logic for how to parse something.

It's typed so you can you can map it.

You can say with a with a client validation that is actually going to transform the data

or could could fail and not transform the data.

And then it it stops the chain of validations and transformations.

So you can parse, don't validate, but you can build that up in a composable way, starting

with like field text.

And then you can even that could give you a maybe it could parse into a maybe string

and then you can pipe that into field required.

And then you can pipe that into field with validation and now you can transform that.

Or instead of starting with field text, you could start with field date.

And now it's going to parse into a date type or field time or these different core input

We have field checkbox and it knows what that's going to parse into.

So you have that parser.

It knows how to present it.

So now you you have like your view function just receives all of those fields that you

So now you're going to have your first name field, your last name field, your except terms

field, which is a checkbox field.

And now you render that and you just you just call, you know, render field function that

takes that field and it knows how to render it with the appropriate attributes.

But you can also pass in your own custom attributes.

But it's just a custom view function.

So you can render it into whatever type you want.

You can render it with whatever surrounding divs and context and HTML you want.

What if the field is not valid yet?

So if the field is not valid, that's OK, because you don't have a parsed like the checkbox.

It just has a raw value.

So the raw value at the end of the day, it's actually just a string, even if it's a checkbox.

So I'm confused.

So those functions, those presenting view functions, they take the parsed value or the

They get the raw value.

Yeah, then which means that you can call that function whether or not each individual field

And because you need to, because you need to present a form before you have valid data.

So it's important to store the raw data, I feel like, because otherwise you get into

some weird interactions.

So maybe we mentioned this in a previous episode, but for instance, one case that is a bit problematic

is when you want to have an input for a number.

And you store it as a number in your model.

And then one of the problems is like if you, if the user empties the field, then that is

not a valid number anymore.

So what you do is you have, you show the default value, which is zero or something like that.

So then there's a zero that is inserted into the field, meaning that you can't just say

one something, you have to say 10 or things like that.

There are so many pitfalls like that where something seems like a really good idea and

you try it and it doesn't work out.

And parsing things into valid data and storing that as your input seems like a good idea,

you run into dead ends like that.

Yeah, it kind of sucks.

But which is all the more reason that it really belongs in an API.

Because doing it, I mean, certainly you can, and I've, some people have shared some nice

looking hand rolled APIs for dealing with forms and you can get something pretty decent.

You can basically say, put an on change or on input handler with a, you could sort of

wrap something that puts like a, that you give it a setter and a getter at the end of

the day, you're saying, how do I get this field's current value out of the model?

And how do I set this field's value in the model?

And then you take that raw model, which stores the low level data, and you use that to parse

into some sort of result or something and you can present the errors and it works out

But what you lose is, you know, for one thing, something with opinions about how to present

errors keyed to certain fields and how to get the errors for a given field.

And there are these things you can abstract out that are really nice.

It's nice to abstract out things about how to present an individual field with the correct

HTML attributes and how to keep that in sync with the parsing logic like we talked about.

Obviously, I think that having an API to abstract these things is a really good idea.

So one of the things that I was looking at in serving these different form APIs out there

and comparing it to what I came up with is some of the APIs take this approach where

essentially somehow or other you're teaching the form API how to get the raw value from

the model and how to put it into the model.

You could do that by putting a message on each individual event and having a clause

in your update function to set that value.

Or some APIs will just say, hey, we're going to have the same message for all of these,

but give me a function that tells me how to set a value and how to read a value.

We'll use those functions in my message.

So at the end of the day, you're teaching it how to set the raw values and get the raw

That's one approach.

Another approach is to essentially have just low level data.

So if you have, so a field has its raw value and it has the field state.

And I really like the approach of having basically unstructured data that's just key value pairs

as your source of truth, have your model manage that.

Then the form can completely take ownership over that.

And so like for example, the Itake Elm form, that package uses more of that low level approach

where essentially you give it the string key for how to get a field.

And then it can handle writing to that field or reading from that field.

But so it keeps track of this low level data, including the state if a field has been blurred

or changed, things like that.

So it can manage all of that state for you.

And the nice thing about that is you don't need to write getters and setters for every

individual field and handle all of that boilerplate.

And it can nicely handle all of those events for you.

So I really like that approach.

What I didn't like so much about the way that particular library does it is that it doesn't

take it all the way.

What I mean by that is a field has three different types of values it can have.

It can be an empty field, it can be a Boolean field or it can be a string field.

And so when you're...

So when you're rendering a field, you need to do form.getField as string or form.getField

as bool and then use that to render the value.

But why not just encapsulate those details and then just say get this field value.

And you might be thinking, wait, if you're saying get field value, don't you need to

pass it a string?

Which means that that's a string to keep in line.

You need to say get field first and you need to say set field first.

Or when you define that field, you need to say the name of that field.

So it can get out of sync.

And that's absolutely true.

My solution to that is this applicative pattern.

You can, you know, when you say like with field, you pipe your form with all these fields

that you're basically declaring.

When you say with field and then you say, you know, first and you give a field for a

The thing you get in the function that you're applying, the applicative starter function,

that first field that you have has a name and your functions for rendering a form field

So, and you can even encapsulate it so that the only way to render using these input render

helpers is with this field type.

So you can't just pass in a string and get it wrong.

And you don't have to reach in and say first, you know, first field dot name.

You have to pass a field type to the input render helper and it knows how to take the

name off of that value.

So in that way, you define your source of truth for a field is you do have to name each

of your fields.

Yeah, which I think is a good practice for accessibility anyway.

And actually it can help with things like a password manager can detect certain field

names and help you fill in information, which is like something I'm often frustrated by

when it doesn't begin.

Please people let my password manager fill information for me.

It's better at it than I am.

So this is just like a semantic form thing that you should probably do anyway.

As you say, I totally agree.

So the nice thing is you only have to define that name in one place and everywhere else

you're using this sort of field type that you pass in and now you've defined your source

of truth in one place.

So what do you mean with the field type?

Like that part is confusing me a bit.

So the field is, you can think of it as an opaque type that you can pass to a field render

So you say, Hey, I have this first name field, please render it.

And this, the form API also gives you a function to render that field.

It knows the name of that field.

It has access to your form state.

And so it can just reach in and say, Hey, I'm rendering the first name field.

I need to go get that value from this raw unstructured data.

So you don't need to write setters or getters to put it in the right low level place.

You just can have a, you know, the form manage all of that state for you.

It reaches in and grabs it.

It knows the attributes to put on that input field, because you know, if you say field.password,

that information is on that field data type that you're using to render it.

So you can still customize those attributes, I'm guessing, right?

To make it look like however you want.

Yeah, that's right.

You can wrap it in some other construct or.

Literally all it is, is you get a helper function that takes a list of HTML attributes that

it will add on in addition to the baseline attributes it includes, like the value attribute,

the name attribute, the type attribute, if it's type password, type text, type telephone,

and then it will give you HTML.

And you render that HTML and you can put whatever you want around it.

You're just defining a function that renders HTML.

So I think that's pretty, pretty cool.

And so like in this API I'm working on, I'm defining the equivalent helper functions for

rendering an Elm CSS input element or an Elm HTML input element.

At the end of the day, it's actually the same thing.

But it's convenient to be able to add styled attributes to it if you're using Elm CSS.

But you could define a custom module to do that as well, right?

At the end of the day, it's just some data that knows what all of this information about

how to present a field and you could swap out a custom implementation.

So yeah, so I think that, I don't know, I think that that applicative pattern solves

this out of sync keys pretty well.

Like you no longer have to worry about, because I always feel a little strange when I'm doing

this like low level access where I'm getting essentially something from a dictionary that

I don't know if it's going to return something and my keys could get out of sync and you

know, that doesn't feel right.

As an Elm developer, we want to make those things a little more robust.

And I think this solves that pretty well.

I don't like putting things into a dictionary because I would, I like to look at my model

and see what fields are there, which can be quite hard to figure out with a dictionary.

Well, like a dict, I mean.

This approach, the source of truth for that is just, you know, you look at the bottom

of your form definition and it's just a series of pipelines of with field first and then

the definition of that field with field last with field except.

So that's your source of truth.

You look, so you still have one place to look and the form state is this opaque type.

It gives you things like whether you tried to submit the form.

And so that kind of brings me to another area I looked at kind of comparing the approaches

I came up with with these other packages out there.

So in looking at these other packages, there's another problem to solve, which is how and

when do you present error messages?

And so, you know, a common approach would be like, wait until you click the submit button

to present any errors.

I mean, you at least need to show the errors at that moment.

And otherwise you're going to confuse the hell of your users.

Otherwise you've sort of defeated the purpose of like painstakingly building up this validation

logic to present errors to users.

Tell me what is wrong, please.

You better show them when they hit submit.

And you probably don't want to show them errors as they're typing in most cases.

I mean, in some cases you might want to, in some cases you might want to wait until they've

at least tried to submit or until they've at least blurred the field, like entered into

focus on a new field.

At Arrima, our form library is pretty much only doing that actually.

It's tracking what has been touched and whether the submit button has been shown, pressed,

And based on that, show the errors that you get through some configuration and a validation

library that is external to the form API.

That is basically what we do at Hemio.

I actually don't remember if we do any sort of parse to validate.

I guess we do, but not sure.

So I guess we don't.

But yeah, so our form API is basically just about showing the errors at the right moment.

And that gives us a lot of flexibility for the rest.

And I mean, at the end of the day, the needs of this API for Elm pages do differ slightly

from if you're, for example, just going to be encoding it into JSON that you're hitting

into an API, right?

Because if you're going to just be encoding it, then why bother parsing into all these

It doesn't, unless you want to give like a real time preview of something or optimistic

UI, things like that.

Which can be handy, right?

So it really does depend on the specific needs of your application.

But it is nice if you can give it a nice experience for parsing data.

Even in how to show error messages, like I saw different approaches to this problem.

And one of the approaches is to just have a hard coded set of logic for when to display

So if you've submitted, then show the errors.

Otherwise for everything.

And after you've submitted, then once the field gets blurred, show the new error, for

You mean before the form has been submitted, show the errors for the field.

Once you've tried to submit, then changing errors after the field blurs, for example.

Yeah, I think what we do at Humeo, I think that's what we felt was nicer for UX and accessibility.

Don't show the errors when you're typing it.

But as soon as you blur it, or as soon as you've submitted the field, or the form, I

think, then update the error as you update the field.

Yeah, so my thinking on that was, so yeah, I saw the two approaches I saw to that problem

were, number one, just have an opinion and hard code that into the API.

And number two, have some sort of validation strategy that you can configure somewhere.

Yeah, which is what we did.

So what I ended up doing to solve that problem is I just have form state that I expose in

that view function that you define, where you get all of those fields that you can render

however you want.

And so in the form state, the form API I'm working on manages the state for whether you've

attempted to submit.

And also each field, you get the state for whether that field has been blurred or changed,

So between that kind of global form state and the individual field state, you can choose

to present errors based on that state.

That's the approach I've taken.

So you can just write like a little error rendering helper that renders it however you

want to render your errors.

In fact, your errors can be any type.

It doesn't have to be string.

There's like a type variable for your error type.

And you can present your errors however you want and whenever you want based on the form

and field state.

That's the approach I went with.

Seems like pretty flexible.

And then people can have an opinion saying, oh, wait, OK, we want to show the errors as

soon as it's been blurred or whatever.

It's totally customizable.

It's kind of like what you talked about the last episode.

You're building on top of a layer and you can only do whatever that layer allows you

So you make it more general and people can do whatever they want on top of that.

But they can only do whatever your platform is giving them.

At the end of the day, what I've really been thinking about with designing this form API

and considering other approaches is what opinions should I have about forms and what opinions

should I let the user have about forms?

And so I feel like the only ones that you can enforce are the ones that are proven to

be better accessibility wise and the rest you kind of have to let them choose.

There's also the question of if I have an opinion on this, can I do something useful

Something useful being improve accessibility, make it safer in terms of wiring or preventing

things from going wrong or getting out of sync.

More guarantees.

More guarantees.

So if you can't do something with an opinion, then why have it?

Yeah, at least as far as API design is concerned.

So for example, you know, if you have an opinion about errors, if you say fields can have errors,

using that opinion, you can abstract away details about associating an error with a

field because you have a concept of that in the API.

There's a cost to having that opinion, but it seems pretty reasonable.

If you have opinions about now the thing is the browser has certain opinions, so why not

take on those opinions in your API?

So if you know the browser has opinions about, you know, if you go to the MDN docs, it's

clearly listed out that there's like a finite set of currently available field input types.

So all right, let's have opinions about that.

We know what kind of data they parse into.

So using those opinions, we can abstract away some of those details, give you more guarantees

So yeah, so that's been like a guiding principle is make sure that you get something out of

having an opinion and basically have opinions about things that the browser also has opinions

Like the browser also has opinions about fields should have names.

Fields should be inside of a form.

So my API abstracts that away.

Fields should have labels as well.

Yeah, although actually I didn't, so far I haven't abstracted that away because I've

essentially said you can present labels however you want.

I guess I could have a simple helper function that says render a label for this, but there

are actually different ways you can render a label.

You can wrap your input in a label.

Or you end, in that case, you don't have to say what that label is for, or you can do

a label for an input element.

Yeah, so that one is going to be hard to, yeah, I guess you could have helpers that

lean one way or another.

Right, so you can have different opinions on that and what does the user really gain

from that opinion?

So I don't know, I might change my approach on that one, but so far my form API does not

have an opinion on that.

Yeah, no, I would at least start with not having one.

At least unless you come up with something nice, but otherwise leave it.

Yeah, and I think as far as accessibility goes, I haven't actually built this yet, but

I'm working on essentially if you try to submit a form, there are some accessibility considerations

for setting focus on the first field with an error.

That's going to affect the experience for a screen reader and also just bringing your

attention to that field.

Yeah, that makes a lot of sense.

I didn't know about that.

Yeah, so those are some things that again, like having an abstraction can do some of

that research for best practices for accessibility and have opinions where it seems like a reasonable

idea to come down on a strong opinion there.

But how do you know which field is first in your form?

Well, the way you build them up in the applicative pipeline, you just say, hey, I'm going to

assume that you're presenting it in the same order that you declared them in.

Yeah, that's an assumption, right?

Or can you inform?

If your view doesn't depend directly that applicative, then it's going to be hard to

enforce, right?

No, you're right.

It gets a little bit odd there.

You also have to consider in practice, how much of an issue is that going to be?

How intuitive is it that you?

I can't believe that I'm hearing those words from your mouth, Dillon.

No, I mean, we talk about this a lot.

We should cover every error.

We're a library and framework authors.

We should do everything perfect for the user.

Yeah, I mean, if you take control over too much, then you can become rigid and take away

the user's ability to do things in a custom way.

I really strongly believe that the way you render the view should be pretty unopinionated.

Maybe an Elm review rule if you're checking the support.

Yeah, I was thinking of Elm review.

But that also has some...

That would be a pretty easy one, right?

Well, I guess if you look at control flow, if they've abstracted it away to other...

Yeah, that's what I'm thinking as well.

We should try to enforce...

To make sure that things work as much as possible to the extent that it's possible.

And here it feels like it's a bit hard to do.

But I would say it's a bit scary to make an assumption that something is right.

Unless you write it in documentation, this has to be in the same order that you display

And you do have to make certain trade offs like that.

But I definitely experienced...

Like I mentioned, my initial prototype did have this approach where you build up the

view and the parsing as you go along and a form and a field are the same thing.

And you sort of combine different sub forms together.

And in that scenario, you do know the order because you're just putting one after the

And that's what I feel like a lot of forms in React land did.

But it also ties you up with how do you display things and how, when, and then you have a

new DSL, which is tricky to make things work with everything.

Like two bridges.

And it's not a nice way to, if you say I want to have a display group and I want to have

some visual thing on the side here.

And so now you're appending parts of the form that actually is just parsing to unit type.

And why is it parsing to unit type?

Well because it doesn't actually represent a field.

It's just a visual element and it's appending it.

But then you want to change your view and you want to put some other form elements next

And it just feels like the wrong abstraction for building up a view in this sort of composable

combinator way.

So it didn't work for me.

I wasn't feeling it.

So then there's another piece, which is how do you do dependent validations?

So you have password, password confirmation, you have a check in and a checkout date and

the check in must be before the checkout date.

Or you only have to check this field if this checkbox is checked, something like that.

So I came up with two pieces to help with this.

So first for the sort of dependent validations.

So I initially, you sort of alluded to how in this pattern I'm using for the view, if

one of the individual fields doesn't parse, how do you present that when it fails?

Because you need to show a view no matter what, whether everything is parsing correctly

or not, whether there are validation errors or not.

But you're showing, you're giving the raw values, right?

You're giving the raw values.

You actually can give like a value that might be parsed or might not be parsed, right?

And no, or not right.

Like I don't get it.

What do you mean?

What I mean is I cannot call a function whether or not the form is valid with a parsed value

because parsing might fail and then I can't call the function with a parsed value.

But if I say, hey, I'm going to parse this date field.

Here's a required date.

The type it parses into is not a maybe date, it is a date.

I'm going to give you, so I can't give you a date, but I can give you a possibly parsed

date value and you can check if it's parsed or not and you can use it if you want to.

So you're giving both the raw value and the parsed value?

Well usually you don't want to use the raw value.

So that was one of the things I was trying to avoid in my design is I didn't want the

user to ever have to deal with the raw value.

And that's one of the things in surveying these different form parsing APIs is that

I did see some APIs where the solution to these sort of dependent validations is, hey,

you can define something that gets access to all of the previous raw values and then

you have to sort of reapply your parsing logic and hope it doesn't get out of sync or hope

that it's fine to compare the raw value.

So you're talking about getting the parsed values in the end then or in the decoding

I mean, you could have it in the view if you wanted to.

It's in case the user wants to do something with it, in case the user wants to...

Pan a match rather than reparse something.

You kind of like take a, I don't know, some value you decode it into and render it next

And you say, hey, I was able to...

Not only was I able to parse this value, but you put the word tomorrow next to a field

because you've set it as tomorrow or you've set it as today or you put the text you're

checking in today because you selected today, you have access to that parsed date.

It's a possibly parsed date, but you check if it's there and then you have access to

So you sort of may as well give access to that value in case you want to use it somewhere

But you can also use it in a function for doing like dependent validations.

And so now, so at first my approach to this was I was only calling this function to do

these dependent validations if everything parsed correctly.

And then I can give you all of these parsed values and you can sort of do these dependent

validations and combine it into a nice type given everything that parsed into nice types.

And the problem with that is now if you're going through the form and you want to get

this nice error message next to your password confirmation that says this doesn't match

your password as you're typing, so you know whether or not you're good.

Now you only get it if you filled the rest of the form.

So you have something that is a required date field or just a required string or whatever

Now it's not possible because it was unwrapping that maybe type when you did the required

or whatever validations you did for individual fields.

And if any of those have any problem, you don't get that error, which is not good, right?

So you should be able to have fine grade control over when you display these dependent validation

errors rather than being locked into some arbitrary thing that's based on convenient

So what I came up with is a validation API that lets you take these like validated types

and combine them.

So you can say like validation.map2 and then you can take the password and the password

confirmation and you can do validation. and then you can add a validation error to an

individual field which takes the password confirmation and associates an error with

Again, it's type safe because you're not just passing it a string password confirmation

for the name of the field, but you're passing it a field.

So the only way you can associate an error is by giving it an actual field that you know

you're using on your farm.

So that's a lot of code.

We're talking about a lot of ideas here, but I'm pretty happy with how that turned out.

So just to get it right, because I'm not sure that I do.

If you use like map2, you're going to show the errors for every individual one of those

two things or map3.

For every individual one of those three things, regardless of whether the other failed.

If any individual field's validations fail, that error will always be shown, no matter

Yeah, I think that makes sense.

But in addition, you can take all of those fields and you could say, hey, I need to check

these two fields to present possible errors.

And I don't care what other fields are parsing or not, but if I'm able to parse this password

and password confirmation, or if I'm able to parse this check in date and check out

So meaning if you've entered a check in date and you haven't entered a check out date,

it's not going to run that.

Because if you do validation.map2 with your check in and check out, well, one of them

doesn't parse into a date.

It's not going to run that dependent validation.

But if you do a dependent validation on those two, and now you have two dates, you can check

that the check in date is before the check out date.

And it will run those assuming that those two fields parse, but it doesn't care about

the rest of the form.

Yeah, that's pretty nice.

And then you can use all of that to build up the final parsed value.

So if you want to parse, don't validate into a nice type, you say, hey, we don't need the

password confirmation for the parsed value.

We just need the password because that is the data type we want to parse into.

But we're only going to parse into that if there's a matching password, you can do that.

And in fact, you could even parse in, you could even recover and parse into a type by

defaulting values if any of the individual fields failed to parse.

And that's nice because you might want to like show a preview or optimistic UI or well,

not optimistic UI if there's not an in flight submission because it's not valid.

You're not going to have an in flight submission, but you could render a preview of something

while something is so you're updating some, you're creating a new product listing and

you can show a real time preview of that product listing.

Even though you didn't enter a price and the price is required, you can default it to zero

or and show that in the product listing.

So you can have a default that's still forbid it from being parsed for real.

So there's a few layers to this.

There's a parse value.

There's a parse value with defaults with fallbacks.

There's validation errors and there's things that are not, have not been filled yet or

can't be parsed.

And the parse values with fallbacks are actually all it is, is it's just this kind of combining

function tells you how to like parse given all of the individual fields.

You can check if they have any errors and you can combine them together.

If that whole process succeeds where there were no errors on individual fields and you

didn't add any dependent errors in addition to that in that function, then you successfully

But you can still parse into a value even if you added errors associated with fields.

And so you can say, you know what?

Yes, I want to present all of the errors with individual fields, which your view function

But I also want to show whether or not there are errors.

I want to get a parse value if I can so I can show a preview.

So you can still do that.

Now in the this Elm pages API I'm working on for receiving the incoming request.

So you submit the actual form data.

It receives the form data.

You reuse that same form parser function and you just don't want to get that value at all

if there are any errors or anything went wrong.

So in that scenario where you didn't give a price for a new inventory, it's like I don't

want to parse into data.

I just want to immediately give an error and say, no, this isn't going to work.

So now I feel like the question is, are you going to publish that as a separate package?

I've gotten that question and I have been thinking about that.

I certainly I'm pretty happy with some of these designs I came up with for this API

and I want to shake things up.

I would love to like influence the way we approach form handling in Elm.

And so I don't know, I haven't decided yet what that's going to look like if it's just

writing some posts about my findings and sharing the Elm pages form API I build, or if it's

going to look like me building a separate vanilla form parsing API that's independent

of Elm pages, or if the two will somehow be connected that I'll have like my own pages

form parsing API use my vanilla one and build off of it or something.

So those are different paths I'm considering.

But I hope that maybe I can sway some opinions about some best practices for handling forms

Yeah, let us know if you want to have this as package or...

Yeah, I might just play around with it to see how that feels.

And I mean, if nothing else to showcase my ideas about forms in a way that you can easily

share on an Ellie and play around with.

So yeah, so the Elm pages form API does have like, like I mentioned, certain coupling to

Elm pages features like data sources for doing server side validations.

And it also hooks into state.

So the Elm pages framework manages the state for you.

So we talked about using low level form state, but the Elm pages framework, this new version

will actually keep track of the client side form state for you.

So you don't need to wire anything into your model and it can hook into that.

And it can keep track of in flight submissions.

And you can use it so you can run your same form parser on the in flight submissions.

In flight submission.

If you press the submit button, it's valid, it's sending it to the server.

That's an in flight submission.

It hasn't come back with a response yet, but you are submitting a new product.

So you know, if you're submitting a new item to your to do list, you can show optimistically,

you can show that new to do list item, or if you're deleting a to do list item, you

can optimistically gray out that to do list item.

So you can reuse that same parser for the in flight requests, the client side requests,

like the client side state and the server incoming server request parser.

So yes, it is going to be like coupled to those things.

But we'll see maybe I'll maybe I'll build a version that's decoupled from that as well.

I do feel like forms are one of those areas where it is nice to have an abstraction to

have a consistent thing across your application.

But it is something that if you want to get started, you can just wing it, build it yourself

from scratch every time.

So if you unless didn't build a really great package for everyone to use with or without

some pages, I think that if you want to get started, you should probably be build your

The only thing I'm wondering about is accessibility.

Like if you have a package that makes accessibility better, which is going to be hard to think

about yourself, like all the best practices, then maybe it's worthwhile.

But otherwise, I feel like, yeah, just to get started, build your own version, none

of a general thing.

But like you have this form, just do a manual model and update and view.

Yeah, I mean, you could definitely start there.

I mean, if you're not doing anything too complex, but if the problem then becomes if there are

many places where things can get out of sync.

So like how many places do you have to define logic that you could get wrong?

How many points of failure are there for things like defining your validation logic?

Could you forget to run a validation for a field?

Could you forget to show the errors?

Could you show the errors for the wrong field?

Could you set a wrong field?

Could you forget to wire in a setter?

But in practice Dillon, like how often does this happen?

And also it's like how much cognitive load is it taking up as you're working on these

things, making sure you didn't get any of these details wrong.

So I personally have convinced myself that we can build a pretty nice abstraction using

low level form state and an applicative pipeline.

And parse don't validate.

Parse don't validate.

I also didn't mention like, but the individual field API for building up a field that's composable,

you start with field.text and then you pipe that to field.required.

You can pipe that to field.withvalidation.

That uses a phantom builder API, which works very nicely for that use case because you

know, you, yeah.

So I'm very happy with that.

I'll share links to these preview APIs that people can take a look at if they're curious

to see more details.

But I feel like this is a combination of a lot of things that we've talked about.

There are big types, build a pattern, applicative, phantom types, using the platform.

So I've convinced myself at least that I think there's a nice abstraction we can build here.

I hope I have convinced some others, but I'll have to put it out there and let people get

their hands on it and see what they think.

It definitely sounds interesting to me, but I'm not sure I could build it right now with

my understanding at least.

So yeah, either you're going to have to write about it or publish it for me to try it out.

There were a lot of details that were really tricky to figure out.

Even if future me went in a time machine and told me, and I listened to this episode about

these sort of design considerations and dead ends, of course, it's hard to learn those

lessons without trying it yourself.

But it was yourself from the future.

What are you talking about?

Time travel, kind of like a time traveling debugger.

But I still had a lot of really tricky puzzles to solve.

One of the things that I will say is definitely challenging with using this API is the types

As you know, working with phantom builder pipelines, it gives you error messages for

nice things, but it can be tough to figure out how do I make this API happen.

It can feel a little cryptic.

And also, if you're building...

So there's some things that are similar to the mini builds alam codec API for building

custom type codecs.

It's using that kind of API?

Yeah, where you have this sort of applicative thing where you declare each of the fields

and then you use those in this function, right?

It's a bit tricky, but the error messages can be very cryptic.

And even for a very seasoned Elm developer, it can be hard to know what went wrong if

you mess something up in terms of the error messages.

But once it compiles...

So there's definitely a trade off there, but I feel pretty good about the set of trade

I mean, you're rarely going to say, well, the API is great, but I'm going to just drop

everything because the error messages are a bit hard to read.

Like yeah, totally.

I mean, if we can improve the error messages, definitely.

But yeah, there's never going to be a...

Well, not never, but...

Yeah, if the API works well, then that's what is most important.

It would be interesting to know whether we could have configurable errors, like tell

the compiler, like, hey, if we have an error message with this phantom type, then show

this kind of message to tell people how to...

Like give links to examples or something or to an FAQ.

Whoa, that would be cool.

It sounds like an amazing idea actually.

Yeah, that's definitely a challenge.

And it's one that I run into, like when I'm building codecs using Elm codec, it's like

when I build it and it's compiling, I'm pretty convinced that it's going to work and do what

But when I'm building it up, if I get an error, like I basically have to be very careful to

incrementally build it up and have it in a working state at each step.

And often I use like debug.to do.

Because like if you forgot an argument in that Lambda that you have for every field

in your codec, the error messages would not help you understand that that's what happened

So build your...

Build it one field at a time.

Yeah, build it one field at a time.

And I'll often just like put debug.to do somewhere.

So it's just like, all right, I want to focus on this part of the error now, get that right,

and focus on the next part of the error.

We haven't described how the Elm codec API looks like, but I feel like this is not the

Well, we do have an episode on codecs and we'll drop a link to the docs as well with

a good example there.

Well, this may be our wonkiest episode yet.

We really got into a lot of nerdy API design details here.

I feel like the fancy build pattern was also quite wonky.

But at the end of the day, hopefully all these wonky details go to serve a better developer

I'm looking forward to a blog post or a package, Dillon.

I'll work on that.

For the release of this episode, please.

I'll do my best.

I've actually been thinking about doing a blog post comparing some concrete examples

of these different approaches as well.

Because I think it's a good exercise in API design and also for posterity, it's nice to

know what kind of tradeoffs there are for these different form API approaches.

Well, until next time.

Until next time.