elm-open-api with Wolfgang Schuster

elm-open-api with Wolfgang Schuster

Wolfgang Schuster joins us to discuss generating type-safe SDKs using elm-open-api.

PublishedNovember 20, 2023

Episode#95

Wolfgang Schuster (github) (twitter)
elm-open-api (NPM package) (Elm package)
Akita (now part of postman)
JSON Schema
dillonkearns/elm-form
Wolfgang's Effortless SDKs blog post
GraphQL Custom Scalar Codecs feature in dillonkearns/elm-graphql
elm-units package
Open API Generator
swagger-elm
elm-open-api Real World example

Transcript

Hello, you're in.

Well, typically, when we have a guest on, it's nice to give them a bit of a break and

let them rest a little.

But recently, we had Wolfgang Schuster on, and I thought, what better way to let him

rest than having him back on the podcast to talk about rest endpoints and auto-generating

APIs with Elm Open API?

Wolfgang Schuster, welcome back to the podcast.

Thank you for having me back.

Great to have you.

That was a good one.

Yeah, this was, again, one of those episodes where I'm like, hello, you're in.

And I'm like, oh, no, he's going to make a pun now.

Prepare for this.

And this was a good one.

This was a good one.

You actually surprised me with it.

Surprised you with it actually being a good one or with me doing it?

Let's say both.

I didn't even see it coming.

I was like, rest.

Oh, oh, rest endpoints.

It does feel like we're giving a little balance to the universe, getting non-Martin guests

a chance as well.

So thank you for representing the non-Martins.

So, so Wolfgang, you recently had a big release of Elm Open API.

And why don't you give us a quick intro to what it is?

Elm Open API is a combination Elm package and Node CLI tool for generating, as you pointed

out earlier, rest endpoints for Elm.

The idea being kind of, I want to talk to some third party service or maybe an internal

one and I need an SDK and no one has made an Elm one.

So hey, now I have an Elm one.

What is an SDK?

Software development kit, technically, I think.

Basically a way to talk to series of functions, endpoints, whatever to talk to some other

service or, yeah, yeah.

In this case, some other service.

So yeah, let's, maybe let's give a definition of what Open API is.

So it's a specification.

What does that specification tell you?

It tells you which URI endpoints service provides, usually slash API slash products or slash catalog

or something along those lines.

Incontain version information, whether it's a get post, put, delete, et cetera.

What body needs to be passed if you're creating something, if you're creating something in

your catalog, what do you need to pass in?

If you're retrieving a list of catalog information, what does that catalog information look like?

How is it structured?

What optional things you need to send in?

What security requirements, headers, bearer tokens, et cetera.

It's a way, it is in some ways kind of like GraphQL schemas, if people are familiar with

It is a similar type of description, but for traditional rest endpoints.

And so because it lives separate the API itself, right, or at least it can.

So you could take an existing popular API and then create a JSON file that describes

that API using the open API specification.

Is that correct?

Or you can go the other way too.

There are tools to generate backend services, backend servers.

So you have a specification and you're like, I need this to talk to my database.

You can go that way.

There are other things that are like PostgresQL can generate one of these schemas for you

in JSON or even YAML as well.

And then you can then generate an SDK from your Postgres schema and have Elm directly

talk to a Postgres database.

There's a whole bunch of different directions you can go with it.

There's even a, there's a cool tool company called Akita.

I think they got, I think they're now part of Postman.

But they, they provided a way to watch your traffic from over your network and generate

one of these schemas from that.

So you could eat, you don't even have to hand write it.

You could just watch network traffic with the, the Postman tool, generate a schema, generate

an SDK, which is kind of a funky, cool little thing.

And then you hope that everyone was using your API the correct way and that they were

using everything.

You always help a little bit.

You never know for certain.

There's always, there's always education.

I'm, I'm sure there are things that are neatly described by an open API specification, but

then it suddenly returns HTML instead of JSON in the response and definitely doesn't conform.

We've all been there.

There's also companies like GitHub who, who define their own schemas and instead of an

octet stream, define an octocat stream, which is not an actual thing, but hey, they have

it returns octocat for you.

It's fun unless you actually need it.

Well, it is pretty cool.

The, for, for anyone who hasn't seen open API specifications or, or use them in any way,

it's more than just saying that this is a string.

Like you can specify that something is an enum, you know, even if you're not defining

types for your schema, you can just say, well, these are the four possible string values.

This response can return or this is an integer in this range and things like that.

So you can actually put constraints on both input and output values.

So it kind of sits on top of or sits next to JSON schema.

And so JSON schema is used to define all of the, the, the bodies essentially.

So those limits, like you mentioned, like if you're requesting price information and

that price has, you don't want negative prices, you could set a minimum, minimum of zero and

that's using JSON schema definitions, which is kind of fun too, that it's building on

top of these prior art and existing things, existing tools.

And then there are also JSON schema validator tools in Node.js or other ecosystems where

you can actually verify before you consume the, the data, the request data, you can

verify that it conforms to that specification.

Which is also what made my job easy is I used Elm JSON schema packages that already existed.

I got to build on others work, just like open APIs building on others work.

What did, what did those tools help you do?

Instead of having to write a parser for everything, I only had to write a parser for or decoder

parser for just the open API portion of the schema.

So all like the body decoders and encoders and everything that's all handled by existing

So it probably saved me, I would guess a few, few good months at least.

That makes sense.

So what are some popular APIs out there that, that use open API?

Well, I don't know how popular it is.

My start was actually with Square.

When I was working at Square some years ago, they produced an open API schema through a

There's, there's was not handwritten.

There's was actually generated.

I believe it was proto buffs.

They used to generate the open API schema and then that was both available to customers

to use as from a development standpoint, as well as through various other things.

So that's where I got my start.

They generated an open API specification from proto buff.

I believe that's what it was.

If I remember right, yes, various teams write their own proto buffs.

Those are merged into one giant proto buff essentially, which then generates an open

Is it like a proto buff specification?

Cause I have never used proto buff.

The only reason I know about proto buff is about, is because Evan talks about proto

buff in relation to Jason and all that.

And was like, Oh, okay.

Proto buff is a, is like Jason with different semantics.

It's like, so how do you make a specification from Jason?

You get, you have a Jason specification of the, sorry, as a open API specification.

You get what I mean.

It's just a series of specification transformers.

Essentially you're going from one specification to another format to another format and eventually

you get SDKs and can write code and yeah, send data.

So that was, that was my first experience.

I had seen it before them, but that was my first like real hands on Leonardo.

Most people in the, in the Elm community know him as mini bill.

He actually helped me a ton with the package and both the Elm package and the CLI tool

He was using it for Spotify.

So they, they have their own open API spec.

He was using it.

I have never actually seen what he built with it.

I know he was building some Spotify tool for himself though.

So and it works for him, which is awesome.

That's the whole reason I built this for, for people to be able to use it.

Stripe I know from my time at square stripe being a competitive squares, like they have

their own open API spec that people can use.

I know there are lots of others.

Most, I feel like most companies these days have some type of open API spec.

I remember at my, at one of my previous companies, we, we had an open API spec or swagger spec

because that's what it was called back then, right?

And yeah, it was only for internal use, but that helped us make sure that the back end

and the front end were in sync and having the spec also allowed for automated tests.

So that was quite nice.

Did you build any back end with those specific, with those specifications as well?

I have been dreaming that somebody will use it on something like Lambda era for their

back end, because a lot of these times these SDKs are more designed for back end because

you're holding API keys and other secret keys that you don't want to expose to the client,

especially payments.

Payments is a great example.

If you were connecting up to Stripe, you're given an API, a secret API key that is just

for your company.

You don't want to expose that to your customers to the front end.

So you put that, you know, in your Lambda era back end and can do your communication with

Stripe from Lambda era back end and present some type of interface to the front end.

And yeah, there you go.

So I don't have the business ideas yet to implement that and use that, but I know it's, I know

there's something there.

That's very cool.

I've heard too that the ChatGPT plugins are built around open API specifications, which

is really confusing because the company is open AI, but they use open API, but they're

And also there's nothing open about open AI anymore.

All that aside, it's...

I've also been confused about reading open AI and...

Like in my mind it's like open API because I was researching this episode or the other

Gets me every time.

And yet they actually are related because ChatGPT uses open API specifications to basically

build a plugin.

You give it a specification that tells it how to interact with your API and then use

plain English to describe how to use those endpoints.

So can you generate an open API spec using open AI?

You totally can and it's really good at that too.

Short circle, but...

See, could use open AI to talk to Lambda era back end.

Wait, does Lambda have endpoints, back end endpoints?

I don't know if they're still in the labs testing area or if they're released fully or not,

but it does have a way to communicate through REST to Lambda era back end.

Makes me wonder now, could you use open AI or co-pilot or something to look at your

Lambda era back end code to generate an open API spec that you could then feed back into

open AI to communicate to Lambda era in production?

Snake eating his tail.

We are in the inception part of the episode.

It's fun though.

I like the code gen side of things.

With open API makes for a lot of, I won't even say code, just generation of information,

Makes it really, already messing it up.

Makes it really easy.

Just go to Swagger.

So yeah, those who aren't familiar with open API or only familiar with Swagger or confused

Swagger is open API version two or lower.

I don't know the history there, but they changed names between version two and three.

They didn't have enough swag.

They lost all their Swagger when they became open.

But they were open about changing the name.

So you mentioned, yeah, that it was for V2 and below.

And when you open the Elm package you wrote, it says here are all the features we support

from V3 and V3.1.

And V2 is in the works, and maybe V3 is in the works as well.

I don't remember.

When I have time, I work on it.

I don't even know why there's a why there.

Is not supported yet.

What is hard about supporting that or is it just like it needs to put in some work or

is it like a very different system?

Not very different.

No, just time to go back through their docs and see what changed between the two versions.

They are mostly compatible versions.

I haven't heard of anyone with like 3.0 not being able to use it.

I mean if you go back to Swagger, any of the two 2.x versions, you might have to do a few

tiny changes, but they're mostly compatible.

So there's the CLI and there's the Elm package.

The CLI is there to generate Elm code for people to use the SDK.

Or I guess it is the SDK then.

Or yeah, to use the rest, the rest endpoints.

What about the Elm package?

Is that meant to be used by people or is it just like you built the parser for the open

API spec and you thought it might be useful for other people as well?

Should I ever look at this package is what I meant?

I do think there are benefits to it.

So going back to Square, all this came to me as something I wanted to work on when I

was at Square because of how we used it.

So I mentioned earlier that we would generate this open API spec.

But what happens from it there?

We didn't just give it to users.

One of the things we would do with it was send it off to another company called API

Matic to generate SDKs.

So that's kind of where I was like, oh, that's really cool, but they don't make an Elm SDK.

I want an Elm SDK.

I'm not going to pay them to make one.

I'll just do it myself.

So it's like, all right, well, in order to do that, I need to be able to parse the spec.

So that's kind of where the package started.

And then in addition to that, we would also take the spec and we would generate documentation.

So if you go to Square's developer docs, a significant portion of that is run through

We take the spec and they would add markdown and other documentation into the spec written

by tech writers.

And you have a website, basically, then from that.

So if you wanted to, if your company exposed an open API spec that other people could use,

you could, in addition to that, generate all of your documentation from that same spec,

which is quite handy, then your documentation always matches your SDK, always matches your

It seems like a, I mean, if you're using REST endpoints, it seems like a great way to go

to get a lot of the tooling benefits that GraphQL would provide.

It seems like a great alternative to GraphQL.

And of course, GraphQL has its share of tradeoffs as well.

The whole overfetching, GraphQL is trying to solve overfetching and N plus one database,

you know, N plus one queries from the front end, but it makes it really hard to avoid

N plus one queries in the database because you have to tack on all these additional queries

and use creative ways to like aggregate them into one sort of query runner, which is very

difficult is a difficult problem.

So, so some people like using REST APIs for their back end.

And if you do, why not use open API and if you get a nice specification, it's way nicer

to use from, from Elm.

Also not everything goes through a get request with, with GraphQL, everything goes through

get request, right?

Or post or post request.

Oh, post request.

Yeah, I think it's two either.

But yeah, usually people do post request with a body.

Wait, you can do either.

Can you also do it with a delete request?

If you're a psychopath, then yes.

No, I think the comparisons to the GraphQL, I think are, are very warranted.

It is very, very similar in many ways to, to working with GraphQL.

You get a lot of similar tooling, a lot of similar typed benefits in a way, a lot of

similar guarantees.

I think I haven't worked with GraphQL long enough.

I've worked with REST for over a decade now.

I've worked with GraphQL for over a year.

So I feel like I still have a lot of learning on how to best use GraphQL.

I think with REST, it's a little bit more fixed in terms of what to expect, which is

There's also, so other, other ways, since we were talking about, would, would you rune

use, use the Elm package?

Another thing I've been wanting to explore and bring it back to beginning with Martins.

I know there is a Martin already exploring this.

Oh, I'm trying to, I'm blanking on his last name.

Wait, another one?

We were talking in Elm, the Elm online meetup a few weeks ago about OpenAPI because the

company works for is using it.

They're actually using the package, not the CLI tool.

So you can too.

But they're looking at doing form generation.

So because it's using JSON schema under the hood for the bodies of the posts and gets

and deletes, you could just use whatever else exists for JSON schema and you could do form

I think he is actually using Dillon, your library as well, for the form side.

So he's kind of combining the Elm Open API and Elm forms to do form generation.

All of these schemas definitely, you squint your eyes and you're like, hmm, these do look

Yeah, in the blog post you wrote about effortless SDKs, which introduces Elm Open API.

You do talk about forms.

So is this something that people can do already or is this something that you want to work

Something I want to work on later.

When I was at Strange Loop, I was it was chatting with someone who they know they're Martin.

I want to say if my memory is not failing me, I believe his name was Kevin.

He was doing some biomedical stuff prior and doing some form generation and had been using

ClosureScript for the form generation and was just trying to explore what else was out

His back end was in Rust.

That's how the machines were communicating with the web interface.

And so he was exploring what else is out there.

I was like, you know, I think there is something Elm could do there.

I don't know for certain, but I think there is something.

We got to chatting about JSON schemas, which does most of the data portion.

And then there's also UI schema, which is UI definitions for rows, columns, and a bit

of other stuff to use in conjunction with JSON schema for defining forms.

So there is there is an existing schema for defining forms on top of JSON schema.

So I think I think I could probably my my idea is to try out something in that area.

I don't know what it looks like yet.

I haven't haven't had the time to get around to it.

But I think there could be something there.

So a little bit confused.

Can you define all the necessary validations on data through the open API spec?

Or is it too limited?

I'm guessing you can do relationships between two objects or whatever.

But relationships are not certain.

You might be able to.

I'm not 100% certain offhand.

You can define so it's not on the open API that's on.

So the validation is done on JSON schema, which is confusing.

But yeah, but open API spec is JSON schema also.

It uses JSON schema for defining request bodies and response bodies.

OK, because you did mention like, yeah, you can say that a price has to be a minimum zero.

So there are some validations.

So I'm curious about like, if I'm using Elm open API, so I've I've run the CLI tool.

I've generated some code for the Spotify API or some open API spec.

And now I've got a folder full of generated Elm code that I can use for all of the rest

endpoints in my specification.

Tell us a little bit about that code.

Is it like what are the what are the functions that it's giving giving you to consume these

What do those look like?

Yeah, it is it is giving you your standard HTTP command requests.

Say you had a create product definition endpoint that so you would give you a create product

definition, it would take whatever data is necessary for actually creating a product

name price, something along maybe a description, optional description.

It would also provide you all the types for what a product is encoders decoders if you

need to use it somewhere else.

It will auto wire up all the encoding decoding for you in the request itself.

But if you need to use it to store in local storage or some something else outside of

the API, all that is provided for you are exposed for you.

That is a pretty cool detail because we just talked about concurrent task in last episode.

And it defines its own tasks, meaning that you could probably not reuse the HTTP request

if you wanted to be able to use them concurrently.

But if you have the building blocks to do that manually, you could still do it, although

you probably still would like to have some kind of cogeneration because it's a lot of

code otherwise.

Don't recall from the episode or from reading the docs.

If you can mix Elm tasks with Elm concurrent tasks.

No, no, not really.

I mean, it's its own thing.

You definitely can't create an Elm concurrent task from an Elm task.

Yeah, so there would be a limitation there because I do I do also expose a task version.

So there is like a there would be a create product, which we're going to turn a command.

And then there's also, I don't remember the naming scheme I use, but it is like create

product task that would create a task as well.

Just in case you don't I don't know what what you're going to want to consume.

Maybe you need both.

So you have the create article, create article task, or you have the decoder article encoder,

you have everything is generated and exposed.

The one thing that is not not released yet, I am working on it.

I think I'm getting close to something that I like is don't make promises on this podcast.

No, no, I am actually decently close, but is is is air handling instead of returning

just an HTTP error.

The the spec or open API specs can also define what type of error to return what the error

messages can look like, which I think is very useful.

That's that is one thing that is.

Yeah, that's really nice to have.

So I'm trying to come up with a better air type that is specific to your API so that

you get this custom air type.

And then if it can't if for whatever reason you're back and returns something invalid

or some other network traffic error occurs that you can fall back to to a regular error

So that's that is the my experiments with that are a little weak.

I've I've been using the real world app for a lot of my local testing and local experimentation.

And that only defines two custom errors, which are there's a 401 on authorized error.

And there is another one called generic error in the spec I'm looking at, which is the error

for everything else.

Yeah, so and I've I've also learned from this that there are actually multiple implementations

of the real world backend in terms of what that spec looks like, which means sometimes

my test work or sometimes my experiments work with the backend that I'm hitting and sometimes

So that's that's my latest or my current current work on the open API stuff.

Yeah, it it's very cool how if a if an endpoint requires an authorization token or a particular

query param with a given name, the code that elm open API generates shows reflects that

so it gives you the record argument to call the function has the authorization token and

the query programs that you need to pass in of the correct types.

One thing I can't help but think about.

And if you're playing Elm Radio bingo, now's the time to get out your bingo cards.

What if you wanted to create some sort of opaque type, let's say,

Ding, ding, ding, ding, ding, please.

I think that was going to be my next question as well.

What a surprise.

Yeah, so like with with Done Kerns on GraphQL, for example, you know, I'm I'm a big fan of

using opaque types as much as possible, both decoding into and using, you know, custom

input types and that sort of thing, like for an authorization token, for example, I might

want to have a wrapped opaque type that is actually an authorization token and maybe

that I can't accidentally pass a string oops, that I mixed up the names of the authorization

token and the username and now that's getting logged to, you know, the console or something

like that, right?

And same with like getting back data, things like enums or things like a non negative integer,

like a price, I might want to have an opaque type to make sure I don't cross wires for

these different result types that I'm getting back and have certain guarantees that I that

I know invariance about those values.

Has that been on your radar at all?

Have you thought about that?

Is that compatible with this approach?

A thousand times yes.

I'm glad you didn't put a no in there because I didn't know to which question it would

have been an answer to.

I honestly don't know which one I would have lined up with.

So the the the tokens that had not specifically crossed my mind, like that specifically, I

do really like that, though.

I don't know if I could make a special exception just for that or not.

I'm tempted to.

So my thinking has been that in this kind of goes with the form generation as well, because

I think it kind of benefits both sides is the schemas.

All these schemas have a way to define your own properties on the schema.

You usually form like X dash custom property name and that can do whatever you want.

Essentially, it can mean whatever you want.

And I think there might be a way to use that to define custom ways to handle encoding and

decoding and maybe form, form handling as well.

So like an example, there would be units.

No one's familiar.

There there is a great package from Ian McKenzie called Elm units.

That is really fantastic for handling anything with any type of unit value, like feet to

meters, you don't want to mess that up.

So why not use a package that just does it for you?

Similarly, if you're dealing with endpoints that are to have that kind of data similar

to a token, like I don't want to pass in the wrong string for a token, I don't want to

pass in feet when I want to use meters in my endpoint.

That would that could be catastrophic in some cases.

There are spaceships that have blown up because of that.

So I haven't had the chance to experiment yet in code itself, but I've been thinking

for months now that there is probably a way to handle automatically pulling in a package.

So a way to find I want to use this package for this type of data and have it auto encode,

decode that type of data, wrap that type of data in whatever wrapper it needs.

Yeah, I really, I think there's a lot of potential in that area, a lot of potential.

That would be, that would be amazing.

So like in the, in the case of Dillon Kern's Elm GraphQL, what I did is I introduced, you

know, this custom scalar codecs file where you, for each custom scalar in your schema,

you define a codec encoder and decoder pair.

And now the nice thing about the GraphQL spec in that regard is you do have this concept

of a nominal type, a type which is not just a set of fields of particular types.

It's like, this is a, you know, a unit of, of feet or of meters, like you can specify

custom scalars and give it a name and say, well, the underlying representation might

be exactly the same as this other thing, but it's not like a, if it quacks like a duck,

it's like this is units and meters, not to be confused with some other unit that we use,

unit and millimeters or something, you know.

So the GraphQL schema definition language has this notion of users being able to define,

to define their own custom scalars.

And then Dillon Kern's Elm GraphQL just provides a way to define encoder, decoder pairs for

each of those custom scalars in the schema.

Now with OpenAPI and JSON schema, I wonder like, is it more duck typed by its nature

in the sense of it's just saying, well, this is an integer and its minimum value is this

and it doesn't have a maximum value.

And you're not really naming things in one central location where you give something

a name and then you reference that named thing you defined at the top.

I will say it's limited by JSON a little bit.

So the defaults are things like a generic number or a string, an array, they're very limited

in that scope initially.

There is a portion of the schema that is just for defining, called objects for, since it

kind of fits with JSON a little bit.

So you, when you define your product, your product could be a ruler.

So a ruler measures things, but doesn't tell you just by the name if it's in imperial or metric,

you have to actually see it or have something else.

And that might actually be defined more in that object schema portion of the full schema,

So there you could, you might have a minimum of like, a ruler is never going to be less

than zero, so a min of zero and a max of maybe 50 or something, I don't know.

You, on top of that, if you wanted to define units, so if you wanted to know if it was

imperial or metric, you would most likely, I think, use a custom property that was unique

Those, those are quite common using custom properties.

It's one thing that is probably the one thing I'm not sure entirely how to support yet in,

in my package or my CLI tools, because they, they can be anything almost that

basically any JSON schema definition can be used there.

Although I do like that you mentioned having a separate config specifically for some of this.

Think I hadn't considered that entirely, but I kind of like that because if I'm using,

if I'm using Spotify's API, I can't modify their, their API spec, their open API spec.

I can't modify it's not my, I mean, I could download it and edit it.

But then when they release a new version, I have to go and manually edit it again.

And that's, that's not fun for somebody.

So having the option to either include your own custom properties or for a third party

spec, being able to define your own local config to add on to it, essentially that

that could be a very nice route to go.

Yeah, if there was some way to hook into that, that would, because I guess what often happens

is like things, you know, tools around GraphQL and JSON schemas, you know, a lot of people

are using them with TypeScript maybe, and they're just like, okay, let's just add TypeScript types.

And they use sort of the lowest common denominator, primitive types to describe it.

And that, and that's pretty nice.

And then with, you know, with TypeScript, you have an enum and it can be five different strings.

And then in TypeScript, you can just describe that.

You can say, this is one of these five strings, which is really nice.

But at the same time, it is, you know, we also like opaque types and being able to have certain

guarantees around those things.

So we also like custom types.

And so if you want to do more nominally typed things, TypeScript isn't, doesn't give you the same

guarantees with that.

And it's not the path of least resistance.

And a lot of tools end up just catering to that sort of, hey, let's nicely describe what

primitives we're using everywhere.

But to me, the really interesting thing is when you go beyond that and you say like, okay,

well, it's giving us all these primitives as a starting point, but then we can actually

describe, give more semantic meaning to that set of primitives and use opaque types around them.

There absolutely is a reason for, like you pointed out, being a being very minimal in your

We ran into that a lot at Square with our open API spec in that we were having to support, we

support TypeScript, which was usually fine.

Ruby was usually fine.

Java and C sharp were always a bit frustrating, more, more due to like the generated definitions

would, you know, the, if everyone's familiar with working with, especially with Java, you would

end up with function calls that have like six nullable arguments.

And so sometimes you just have to pass in null like four, five, six times.

That was always fun.

But yeah, so you don't know, like you're having to potentially target many different languages

with very different semantics.

And it's hard to define something that fits all of them equally.

Everyone's going to be slightly disappointed in their, in their generated SDK.

So if you, yeah, if you have your own little config, you say, cool, I like this, but I'm

going to tweak it to fit my language or even to fit my app, like just to be able to fit

your specific project.

Very nice to have too, too many ideas now.

To throw another idea out there.

It, uh, like, I, well, I'm not sure if you do this at all.

I think maybe you don't, but for enums in a JSON schema, you can describe like an enum

like these are the five possible string types for this value.

Do, do those get generated as string values or custom types?

Believe they get generated as custom types right now.

It was, they do.

I'm fairly certain they do.

I went back and forth on that a lot actually.

Intuitively, you think that's the right way to go.

However, there are, there are weird edge cases, um, which I only know from dealing

with them at square in that people like to sit on one version of an SDK for a very

long time, sometimes.

And so if you, you keep updating your backend to define new, new variations on

that enum, the SDK might not be able to handle that anymore.

Uh, and so we would run into issues with customers on very old versions of the SDK,

not having the time, maybe even to upgrade.

Maybe they're just swamped with other feature work.

They can't, you know, spend a day or a week even upgrading the SDK, but you

don't want to break them either.

So we ended up, at some point, we ended up switching most of our SDKs back to

using strings for enums for that specific reason, which is a bit disappointing, but

it also kind of makes sense from that standpoint.

I do think this did inspire me though.

So maybe that's where opaque types come in.

You could probably with Elm instead of exposing the strings directly.

I wonder if there's some way to do something opaque or just expose them through.

Like if you want to create them, you just expose a function name that behind the

scenes is still generating a string, but hidden within some opaque type.

So you could totally do that.

And, and the benefit to that would be that, uh, from an API point of view, of

course, if you're not actually publishing a package with this generated SDK from

Elm Open API, then the, the breaking changes thing is more of a, you know, you

know, when it's broken because your code doesn't compile, but, but from the point

of view of like publishing breaking changes, you can add an enum variant and it's

not a breaking change with an opaque type because you're exposing a new function.

So that's a minor version bump, not a major version bump.

But if you remove one, then it's a major breaking change, uh, because, because it

is, because it could break someone's code.

Uh, whereas with a non opaque custom type, you're in, we should, uh, we should lobby

to, to call them explicitly non opaque.

We should lobby to call them non opaque types.

Transparent types.

Bad types, the good types and bad types.

It is considered in a published API, a breaking change.

If you, um, if you add a variant, because, you know, you could have a case

expression that you now have to handle that one additional thing.

Now, on the other hand, so, so I'm able to do case expressions is handy.

So that's a trade off too.

But, uh, you mentioned if you remove, uh, a possibility in the API, then that is

also a breaking change for the backend, right?

Because you're not, not, you're now not able to call it to call, get

article with type, uh, some vegetable.

I don't know, like vegetables are not sold anymore.

Vegetables don't exist anymore.

Well, now you can't send that anymore.

And that's a breaking change also.

So that shouldn't happen in that case.

Usually the people go to a V two or V three of that same endpoint, right?

So usually what would happen is you would add a new, new, new vegetable type.

Uh, so carrots for whatever reason, you forgot to add carrots in, in your

initial release of your backend and you go and you're like, shoot, I forgot those.

You add carrots.

That's, that's stupid because that's the best vegetable.

I, I is a fantastic vegetable.

It's probably up in my top three.

But hey, you know, you were busy.

You, you were working 12 hour days for whatever reason to get this shipped.

Cause you're full and work.

Carrots before hands.

So you ship it, a customer goes and installs version 1.0.0 of your SDK and they

go and they use it and they're like, great.

And they don't actually use that vegetable type.

They don't care about it, but it's still part of the response.

So the response decoder has to decode it.

They don't care about it where they don't care it.

They don't care about it.

So they, you go, you add carrots.

It's a non-breaking change.

It's a minor, minor thing.

You've added a new type.

Their decoder doesn't support it.

So their decoder, if it got carrots would fail because it doesn't recognize it, but

they're not actually using it.

So now you've broken their, their, their SDK for no, no apparent reason.

But if you wrapped it in just a very small, slim, opaque type that took it as just

like unknown, unsupported type, they can keep working.

And everyone else who wants carrots, they still have access to carrots.

The end user might still be defining a custom type that, you know, they're

going to be adding carrots to and handling that in their own code, right?

So in a way it, it passes the buck downstream.

So it's, it's an, it's an inherently challenging question.

There's no easy magic bullet to it.

If a new possibility arises, like a carrot gets introduced by the back end, then

your whole front end doesn't compile anymore because you've upgraded this back.

And now the front end needs to real quick handle that case.

I think there's, so there is an edge case here.

If you are working with open APIs internally within your company, odds are you

do want the custom type, regardless if it's opaque or not, you do want, you

don't want to use stringly typed enums.

You want the custom type because if, if your back end does change something, you

want that reflected in your front end.

However, if you are using this between companies, possibly between multiple

companies like, like Spotify or something like that, then the custom type isn't, I

think there's more risk involved with the custom type in terms of breaking the

customers, the customer's application, which you don't want to do.

There is still the edge case that they could forget to or customly handle

carrots or rutabagas or whatever else they decide they need to support or you

forgot to support.

But that's like one of the worst vegetables.

Oh, poor Vegas.

They deserve it.

But yeah, you don't want to, you don't want to break your customers and point, and

they might not have the time.

There isn't the same level of communication that you have between teams within a company.

So maybe that's something that is like a flag maybe on or in your config for when

you generate your own SDK from this.

Maybe I have something that says, I want custom types for enums or I want strings

for enums and let, let them choose or let them override to whatever fits their product

At work, we've had some, we use Elm GraphQL Dillon's version.

And some of the things are using custom types.

And that creates some problems.

For instance, when we do migrations, like the backend migrates to a new version where

it needs to support having older versions or newer versions of the front end talking

to it or the other way around.

Like you've seen Mario Rogers talk like all those variations.

Well, we need to support those.

And therefore we are thinking of like moving those custom types into stringified, stringified

types, which is not great.

But it is a lot more flexible.

It is a lot easier to ignore some things that are unknown.

And you just say, okay, well, the front end doesn't know about carrots.

But if it gets a carrot, what should, what should we do?

Should we just ignore the message or should we crash hard?

And in some cases, it's fine to just ignore it.

It really depends on the use case, obviously, or the situation.

So we also use GraphQL at work.

And I think the way we handle that is we generate a hash, essentially, of our GraphQL

schema or whole GraphQL schema and just upload that as a single file and then use

that as a versioning thing.

And if the front end sees that that has changed, then it will refresh the page, essentially.

Kind of like if you're on Slack on the browser or Discord or many other messaging apps, and

they're always like, a new version is available.

Click this button to update when it just reloads the page.

Or Discord, which will just not let you look at anything if it's outdated, which I get.

I totally get that.

Well, it really depends on the situation, right?

Because if you're on Discord, you're typing a message and it says, oh, I need to update.

Okay, well, let's refresh.

Your message is still in the box.

But if you're editing a very complex form, you don't want to lose your things.

If you're doing something that takes a long time to set up somehow, then you don't want to lose that.

I think it depends on the size of the company, too.

I don't know how big you're at CrowdStrike, right?

And I don't know how big CrowdStrike is, but if you get to the point where you have multiple

back-end teams interfacing with multiple front-end apps and one of them wants to change an enum

and add or remove that enum, that can be very difficult to coordinate across teams.

A lot more difficult than one back-end team and one front-end team, essentially.

Or like one front-end app and one back-end app.

So it mostly depends on who is your customer.

Is your customer the same mono repo application with one back-end, one front-end, and then they're

always in sync because there's always the same person who makes the back-end changes knows how

to do the front-end changes as well.

Or do you have back-end that is not always in sync with front-end or you use web components

or you're shipping part of your products in not the same way?

So the flexibility, right?

Yeah, it's the hard part.

I always like to view it as the hard part is communication.

And if the communication is hard, maybe strings are easier.

Communication is easy.

Then go with custom types when you can.

I mean, you communicate with chat GPT only with text.

Image is nowadays as well, right?

I was also thinking too, other things I've gotten to look at using this with, I know

super base is a back-end service like database slash back-end as a service.

You can also, I believe generate, I think they're swagger specs, but you can generate

those from it, which means you could build an L map on top of super base with generating

all the rest end points or the rest SDKs for your front-end.

I started experimenting with it, but yeah.

Again, have no business business ideas to actually build with it.

Is your Elm open API the first time that a production ready version has been shipped?

I feel like there have been murmurings of this for a long time.

I mean, it's like an obvious fit for Elm because of its types.

There is a, if you're talking open API and Elm, specifically,

there is a Java based one.

Trying to remember, it was pointed out, I might have come across it a while ago,

but someone pointed it out to me recently.

That one, I think that one is, it's from open API, general or open API tools.

They release an open API generator and it can generate Elm.

There's also a swagger decoder Elm package, but not, it doesn't generate code as far

Do you think everyone ever seen that at one point?

I probably did and then promptly forgot about it while looking at other packages.

The, so when the open API tools one got brought up to me, I was very curious to see

how ours differed because I didn't look at it at all while working on my stuff.

It is one thing I do like that it does is it splits out into multiple Elm modules,

kind of around types a little bit.

It splits it out based on, so it splits out your endpoints into one module, your

request body, response body objects into another file.

And I think there might be a third one it generates as well, which is kind of nice

because it breaks things up a bit.

I would like to do the same though, I think more around types themselves instead

of around functions.

I think that feels a little bit more Elm like to me.

I think the open API stool feels a little bit more Java to me.

And then same with, I noticed too, with how you call functions feels a little bit

more Java like and how they're written for it, bringing back a little bit to off

off stuff and tokens.

The API tools generation provide like a with authentication function to allow

you to add that bearer token or JWT or whatever it is to your request.

But it's always optional and you don't know if you need it or not for your

endpoint, which doesn't feel Elm like to me.

Like if this function requires, if this endpoint requires a token, then it's,

you should be forced to pass it in, which is what mine does.

If for every endpoint that requires a token, you're forced to pass in that token.

Otherwise, what's the point in requesting the data because it's just going to fail?

So I think, yeah.

So I think there's something there where maybe I definitely need to break mine

out so that you don't have an API module that's 5,000 lines long, 10,000 lines long.

How do you manage like staging API URLs versus dev URLs versus production URLs?

Is there a way to handle that?

I think there is.

I hadn't actually thought about it too much.

So the way the schema works is there is like a top level URL defined.

So Spotify's might be like HTTPS colon slash slash Spotify.com slash API.

That is then used for all the endpoints and the endpoints are then just slash

playlist or slash artists.

And it all gets joined together.

I believe you can define multiple top level API or yeah, top level URLs.

So I guess I'm not sure what that looks like, but that probably should be something

that's handled.

Yeah, because you could talk to like a real world backend.

But you can just say, well, whatever URL, whatever backend I'm targeting,

the URL is defined not at code generation time, but at runtime or right.

But that doesn't work with your approach as far as I can tell because you generate

that URL inside the you put that URL inside the generated code.

But that could be an argument rights as well.

Yeah, there there likely are a lot of additional arguments that need to be

optional ones, like completely optional.

The real world server might be behind a proxy of some kind or something like that.

And so being able to say with proxy URL or something along those lines would be

a very nice thing to prepend every URL requests that you make with that.

There could be other things as well, maybe cuss, maybe for whatever reason

you want to handle custom decoding just locally in your app.

You're like, the generated coding is great, but I want a little tweak on it.

Maybe in the special case, so like with decode map or something along those lines

might be might be a nice thing to have.

There are other ones too.

I know I want to add retry at some point.

Retry is something that's usually usually nice to have.

For request fails, maybe retry every 30 seconds incrementing by 20 seconds

or something along those lines for 15 times.

Wouldn't you be able to do that through the task API, the Elm task API anyway?

You could write your own retry.

Yeah, but it'd be nice to this is mostly a lot of these things are things

that I knew were request when I was working at Square from users

that were just really nice to have built in for the language.

So why make you write your own retry logic when I can just generate it for you?

Because it's never exactly what I want.

Yeah, I get the feeling too that you're talking Wolfgang about a sort of

pre-packaged SDK, something, the kind of thing that would be like NPM install

Spotify API or GitHub API or something.

And then it installs some JavaScript functions that let you use the API.

So you're talking about that sort of pre-packaged SDK, right?

Because that's, I find it very useful when I when I'm doing JavaScript

and I'm like, I need to talk to the service.

Oh, hey, there's a package I can talk to the service and it's no setup.

You there's a lot like you usually have to pass on a token once your API token

for that service once and then you get some type of pre-pre-setup request.

And then you just say, OK, go get me this data and you're done.

And then if you look at I'm trying to think of some but with those pre-packaged ones,

if you look at their REST APIs and you go to the rest docs,

it's it's a fair amount of work to get back to that same point.

And you're not sure, like, oh, shoot, did I type that wrong?

Did I type the URL correctly?

Did I remember to pass in the off token?

Did did they change their their timeout like or some limits?

Like maybe the API only allows you to make a request every 15 seconds.

Did I actually set that to 15 seconds or did I set it to 10 seconds

because I was looking at something else and got a number wrong.

So having those types of experiences is very nice.

And and I don't expect companies to go make Elm SDKs.

Like that's a lot of work to go and make SDKs for other languages.

So maybe I can help bring that experience, that nice quick.

I want to use a service with minimal effort, that experience to Elm.

Would you so let's say Spotify got big into Elm?

And they wanted to make it easy for people to use their SDK.

Would you recommend that they would generate an Elm package

or that they make their open API spec

readily available and give good instructions on how to generate

it's using your package, like especially with the idea of having

custom opaque types and all that.

Maybe it's better to have it to generate it yourself so you can edit

a bit instead of having a readily but fixed Elm package.

I think handwriting your own SDK for especially a larger API can be a lot of work.

Oh, no, in both cases, it would be using your your package,

but it would be publish publishing the results or telling people to

generate themselves using your tool.

I think it would be I think it'd be publishing would be my guess.

I think that if Spotify were really big into Elm,

my my thought would be you would go to their developer page.

They would have their first listed out their SDKs.

And those would be say then Elm and JavaScript and maybe Python.

And then they would after that list their API spec

and their rest endpoint documentation.

You'd probably still want the rest.

I mean, you'd still want the rest documentation regardless

because it's helpful for understanding the SDKs.

But you would list the SDKs for saying like, hey, we support these.

We put time into these because we want you to quickly get up to speed

and build an app with our stuff.

If you're using a language that doesn't isn't listed here.

Here's another way to go about an interface with our with our API.

Also, if I had one once I have some way to like

customize the generated SDK with it like extra config,

they could go into and like define their own extra stuff that they want.

Like, like, you know, a custom.

I'm trying to think what would be Spotify.

I don't know what Spotify I was going to say track length always

a positive number, but you can have tracks that are negative have negative time as well.

Oh, yeah, great way to hide hide audio in tracks is to go negative.

Yeah. And you can't if I remember.

So I know this from back to listening to CDs

because you can't really do that on a tape.

There's tape has a start and an end.

But in CDs, you have tracks that have a start and stop time.

If you have negative time, the only easy way to get there

is to finish the previous song and it will start the next song at the negative time stamp.

If you skip to the next song, it starts at zero.

And so you would completely miss that like hidden audio.

Oh, is that for instance, like to make the transition between songs nicer?

I maybe but maybe like a translate type thing.

Yeah. Oh, you can hide.

You could hide an entire song there.

I've had CDs back in the day where they had like negative four minutes

and would hide entire songs in a track.

What the hell? Yeah.

I mean, there are songs where the play song, then they have like two minutes

of of nothing and then a little thing at the end.

Is that it as well?

Like and that's somehow good.

That's that's kind of.

But that's not the same because you can like you can easily tell.

Oh, my my track length is listed as six minutes and you get to four minutes.

And then it's just two minutes of white noise.

And you're like, why is it listed six minutes?

Let me like skip forward to it.

But if it's negative, that's not listed.

So the only way to really know it's there is to either like rewind.

Maybe I don't even know if that works.

I think you have to like finish the previous song and let it auto go to the next section.

So so if you're if you're a sucker for shuffling songs in an album,

like, no, you're not going to have it.

I knew shuffle as long as you finish the previous song.

But I don't know if that works in web with MP3s.

I don't know if MP3s can go negative.

It might just be a it might be a forgotten thing, a lost thing with CDs.

I'm very curious now.

I might have to after this, I'm going to look all this up now.

I'm really curious because I forgot.

You just play this.

Yeah. So yeah, no big type for that one.

No, in less in less than MP3s, you can't have negative time.

I don't know. Now I have to kind of hope it doesn't.

It is pretty cool to be able to do that.

So yeah, so they Spotify could go and add their own flavor

onto the SDK generation that really makes it shine.

That if you or I went and generated their their SDK ourselves, we wouldn't have.

So I know we've talked about it during one of many Elm Cogen episodes.

But it's still worth pointing out that it's really cool that in Elm,

when you generate these huge API files, you only get in a ship, whatever you use.

And I find that to be so cool.

But like you just said, like, oh, yeah, like you forgot about it

because you don't have to think about it anymore.

It's just fun. It's so cool.

Yeah, I was when I initially started this out, I was playing with the GitHub

And I think at one point I was getting during some of the experimentation,

we were getting up around like 10 or 15,000 lines of code, I want to say.

Maybe more than it was probably more than that.

Yeah, that's actually not as bad as I would have imagined.

It's not it's definitely not the biggest Elm module by far.

But I don't think GitHub's API is the largest API either.

So I do wonder if there are APIs out there.

There probably are that would that would break Elm if I generated it right now.

I would imagine there are.

I should try and find one.

I wonder if there are any listeners who know of one that is big enough

that would that would break the Elm compiler because the generated code is just too large.

Yeah, if you generate you generate your SDK, you only use the commands.

You never use a task.

That's a third, at least, if not half of the Elm module that is just completely

ignored and never shipped.

Yeah, in your code.

And if you expose all the decoders and you don't use them, that's fine.

And therefore, that's why it's fine for you to expose all those things as well.

And I can't imagine like if you do this in TypeScript, like how how do you avoid

bundling all that code with with your production bundle?

So there's like module that code in a donation.

Like if you don't import a module, it won't be imported.

So you need to split things up by module, but it also uses objects, right?

So anything or classes.

So if you do anything with a product, then that for sure, all the things

that are related to products are going to get shipped.

Like if you're if you're using Spotify, for example, maybe you never hit

put or artists, you never hit the artist endpoint, but every song has an artist.

So you need the artist decoder for every song.

So regardless, that's going to get shipped.

So you want the banana, you get the gorilla and the giant.

I do wonder how that works from language to language.

I should I should generate trying to generate the square square SDK for

Elm and see how big it ends up compared to the other languages.

Because I know they ship like six to eight languages, I think it is.

I'm guessing it's on NPM and you can find it there.

See how big the API is.

They ship a square.

It's just square.

I think square square up square.

It is unpacked.

It is six just shy of seven megabytes.

That's for the node SDK.

I don't know how big they they also have the Java one on Maven.

There's a PHP one Rails C sharp.

So this is really specifically for node.

It's not for front end JavaScript.

That would definitely be a problem for your bundle size.

That's that's a lot.

That does I wonder though, like does that affect I don't know enough about various

languages that have start like like Java has a startup time and does the size of

your dependencies drastically affect the startup time of your server then?

It's a problem for for cold starts for serverless functions.

That's for sure.

It there's a there's a hard limit and it slows it down.

So yeah, so that is an interesting thing.

I never considered that.

So we talked about cogeneration for open API specs.

We talked about cogeneration for GraphQL.

What else is missing in that realm?

Like do we have protobufs, cogeneration that is missing?

Do we have other applications that that are not even related to SDKs?

I think there is.

I want to say there's a protobuf some some Elmstaffer protobuf.

Yeah, there is actually.

There is one for NATS that was just released this week.

I don't know what NATS is yet, but I am kind of curious.

I think there's gRPC is another one.

I think I've seen something for that realm.

There's always a few new ones here and there.

There's one that's message something.

Web message or something like along those lines.

I think most have at least a minimal implementation in Elm.

So the other code gen aspect that I would like to explore someday.

Maybe maybe I'm not sure if I'm going to go to it before or after forms is testing.

I figure if I can generate the endpoints, can I not generate Elm test code as well

for those endpoints so that like if you were right, say you were using Elm program test,

you could send out your requests and have it respond with fake data essentially that is generated.

I've thought about this for Elm GraphQL as well.

Like there's this factory girl Ruby gem where you're using these testing factories to basically say

it knows what kind of data it's generating.

You can let it generate some random things or give specific data in some of those pieces.

But that would be very interesting for sure.

But generating a nice API for it is a challenge.

Also like generating results that make sense can be difficult, I think.

For instance, if you like create a user, then you need to return the same data that you gave to generate the user.

So the same name as the input, the same age as the input, but a new ID and that ID has to be different

from all the ones that you created previously probably, right?

So like because with Swagger back in the day, there was the ability to generate test code like this.

But in some cases, it's not exactly what you want, especially if you want things to be connected in some shape or form through IDs or something.

It gets tricky.

So very challenging, I think.

But if it works, super interesting.

Yeah, maybe I'll do that after form generation.

No, no, no, I'm still motivated to do it.

It's more what given a year, what can I do in a year and what will be the most beneficial?

Testing is still very beneficial, but I think more people benefit from forms than tests is my guess.

I think the testing side is more, I think there's an interesting challenge there and also provides a benefit, whereas forms,

it's mostly benefit.

Like the challenge I don't think is enormous or insurmountable, but the gains that people have from it are significantly higher.

Also, if you wanted to make it work for an program test, you would need to generate a command, a task, and some kind of effect

that goes well with the effects that the user defined.

I think you might just consider having like a

wrapper function where every endpoint calls that function to generate a whatever, whether it's a command, a task, a backend task, an elm concurrent task,

an effect, whatever it might be, like just

let the user define that's using the code gen part,

define a function that takes the decoder and headers and

HTTP method and all that as input, and then turns that into a something,

and then you let them do that. The hard part is in the generated code, then having the appropriate

type signature for all of that generated code, which you kind of want, and that becomes a pain.

It is, it's a type gymnastics, basically.

You have four hours and you are not allowed to use more than three type variables.

Well, the exam starts now.

Well, Wolfgang, this is a really great asset for the community. So, thank you for pushing across the finish line. I really do believe

the more pieces like this we have in the ecosystem,

fewer caveats there are to like, oh, but does the Elm ecosystem, does it have good GraphQL support, does it have good

OpenAPI support? So, thank you.

And if someone wants to get started, what's a good place to start, where can they learn more?

They're completely,

if they want to go down the road of learning OpenAPI, I would say just Google OpenAPI.

Don't confuse it with OpenAI. You and I have been doing it lately.

If you are looking to use it,

check out the NPM package.

Most likely, if you're looking to generate, and then whatever API endpoints you're, whatever service you're looking to communicate with,

generate something.

If you're looking to contribute to new features like form generation or testing, if you're feeling ambitious,

feel free to reach out to me on Slack or Discord,

or however else you know how to reach me. Amazing. And the real world

example that you have where you're hitting the API using

using OpenAPI is also very handy. So, worth taking a look if you want to see what it looks like calling these

generated functions. Thank you so much. Wolfgang, great having you on. Thank you for having me. It was great.

And you're in. Until next time. Until next time.