elm radio
Tune in to the tools and techniques in the Elm ecosystem.
elm-open-api with Wolfgang Schuster
Wolfgang Schuster joins us to discuss generating type-safe SDKs using elm-open-api.
Published
November 20, 2023
Episode
#95
Wolfgang Schuster (
github
) (
twitter
)
elm-open-api
(
NPM package
) (
Elm package
)
Akita (now part of postman)
JSON Schema
dillonkearns/elm-form
Wolfgang's
Effortless SDKs
blog post
GraphQL
Custom Scalar Codecs feature in
dillonkearns/elm-graphql
elm-units
package
Open API Generator
swagger-elm
elm-open-api
Real World example
Transcript
[00:00:00]
Hello, you're in.
[00:00:02]
Hello, Dillon.
[00:00:03]
Well, typically, when we have a guest on, it's nice to give them a bit of a break and
[00:00:08]
let them rest a little.
[00:00:09]
But recently, we had Wolfgang Schuster on, and I thought, what better way to let him
[00:00:15]
rest than having him back on the podcast to talk about rest endpoints and auto-generating
[00:00:21]
APIs with Elm Open API?
[00:00:24]
Wolfgang Schuster, welcome back to the podcast.
[00:00:28]
Thank you for having me back.
[00:00:29]
Great to have you.
[00:00:30]
That was a good one.
[00:00:32]
Yeah, this was, again, one of those episodes where I'm like, hello, you're in.
[00:00:37]
Hello, Dillon.
[00:00:38]
And I'm like, oh, no, he's going to make a pun now.
[00:00:40]
Prepare for this.
[00:00:43]
And this was a good one.
[00:00:44]
This was a good one.
[00:00:46]
You actually surprised me with it.
[00:00:50]
Surprised you with it actually being a good one or with me doing it?
[00:00:54]
Yeah.
[00:00:55]
No, no.
[00:00:56]
Let's say both.
[00:00:57]
I didn't even see it coming.
[00:00:58]
I was like, rest.
[00:00:59]
Yeah, yeah.
[00:01:00]
Okay.
[00:01:01]
Oh, oh, rest endpoints.
[00:01:06]
It does feel like we're giving a little balance to the universe, getting non-Martin guests
[00:01:11]
a chance as well.
[00:01:12]
So thank you for representing the non-Martins.
[00:01:15]
Happy to.
[00:01:16]
Happy to.
[00:01:17]
So, so Wolfgang, you recently had a big release of Elm Open API.
[00:01:24]
And why don't you give us a quick intro to what it is?
[00:01:29]
Sure, sure.
[00:01:30]
Elm Open API is a combination Elm package and Node CLI tool for generating, as you pointed
[00:01:42]
out earlier, rest endpoints for Elm.
[00:01:46]
The idea being kind of, I want to talk to some third party service or maybe an internal
[00:01:52]
one and I need an SDK and no one has made an Elm one.
[00:01:57]
So hey, now I have an Elm one.
[00:01:59]
What is an SDK?
[00:02:02]
Software development kit, technically, I think.
[00:02:04]
I don't know.
[00:02:06]
Yeah.
[00:02:07]
Basically a way to talk to series of functions, endpoints, whatever to talk to some other
[00:02:13]
service or, yeah, yeah.
[00:02:16]
In this case, some other service.
[00:02:18]
So yeah, let's, maybe let's give a definition of what Open API is.
[00:02:24]
So it's a specification.
[00:02:28]
What does that specification tell you?
[00:02:31]
It tells you which URI endpoints service provides, usually slash API slash products or slash catalog
[00:02:42]
or something along those lines.
[00:02:44]
Incontain version information, whether it's a get post, put, delete, et cetera.
[00:02:52]
What body needs to be passed if you're creating something, if you're creating something in
[00:02:56]
your catalog, what do you need to pass in?
[00:02:59]
If you're retrieving a list of catalog information, what does that catalog information look like?
[00:03:04]
How is it structured?
[00:03:06]
What optional things you need to send in?
[00:03:08]
What security requirements, headers, bearer tokens, et cetera.
[00:03:13]
Yeah.
[00:03:14]
It's a way, it is in some ways kind of like GraphQL schemas, if people are familiar with
[00:03:20]
those.
[00:03:21]
It is a similar type of description, but for traditional rest endpoints.
[00:03:27]
Right.
[00:03:28]
Right.
[00:03:29]
And so because it lives separate the API itself, right, or at least it can.
[00:03:37]
So you could take an existing popular API and then create a JSON file that describes
[00:03:45]
that API using the open API specification.
[00:03:49]
Is that correct?
[00:03:50]
Correct.
[00:03:51]
Or you can go the other way too.
[00:03:53]
There are tools to generate backend services, backend servers.
[00:03:59]
So you have a specification and you're like, I need this to talk to my database.
[00:04:04]
You can go that way.
[00:04:07]
There are other things that are like PostgresQL can generate one of these schemas for you
[00:04:13]
in JSON or even YAML as well.
[00:04:16]
And then you can then generate an SDK from your Postgres schema and have Elm directly
[00:04:25]
talk to a Postgres database.
[00:04:28]
Yeah.
[00:04:29]
There's a whole bunch of different directions you can go with it.
[00:04:33]
There's even a, there's a cool tool company called Akita.
[00:04:38]
I think they got, I think they're now part of Postman.
[00:04:41]
Yeah.
[00:04:42]
But they, they provided a way to watch your traffic from over your network and generate
[00:04:48]
one of these schemas from that.
[00:04:49]
So you could eat, you don't even have to hand write it.
[00:04:51]
You could just watch network traffic with the, the Postman tool, generate a schema, generate
[00:04:57]
an SDK, which is kind of a funky, cool little thing.
[00:05:01]
Yeah.
[00:05:02]
And then you hope that everyone was using your API the correct way and that they were
[00:05:08]
using everything.
[00:05:10]
You always help a little bit.
[00:05:12]
You never know for certain.
[00:05:13]
There's always, there's always education.
[00:05:15]
Always.
[00:05:16]
Yeah.
[00:05:17]
I'm, I'm sure there are things that are neatly described by an open API specification, but
[00:05:22]
then it suddenly returns HTML instead of JSON in the response and definitely doesn't conform.
[00:05:29]
We've all been there.
[00:05:32]
There's also companies like GitHub who, who define their own schemas and instead of an
[00:05:37]
octet stream, define an octocat stream, which is not an actual thing, but hey, they have
[00:05:44]
it returns octocat for you.
[00:05:47]
It's fun unless you actually need it.
[00:05:50]
Maybe.
[00:05:51]
Yeah.
[00:05:52]
Well, it is pretty cool.
[00:05:53]
The, for, for anyone who hasn't seen open API specifications or, or use them in any way,
[00:06:02]
it's more than just saying that this is a string.
[00:06:04]
Like you can specify that something is an enum, you know, even if you're not defining
[00:06:09]
types for your schema, you can just say, well, these are the four possible string values.
[00:06:15]
This response can return or this is an integer in this range and things like that.
[00:06:22]
So you can actually put constraints on both input and output values.
[00:06:27]
And that's us.
[00:06:29]
So it kind of sits on top of or sits next to JSON schema.
[00:06:34]
And so JSON schema is used to define all of the, the, the bodies essentially.
[00:06:40]
So those limits, like you mentioned, like if you're requesting price information and
[00:06:46]
that price has, you don't want negative prices, you could set a minimum, minimum of zero and
[00:06:51]
that's using JSON schema definitions, which is kind of fun too, that it's building on
[00:06:56]
top of these prior art and existing things, existing tools.
[00:07:01]
Right.
[00:07:02]
And then there are also JSON schema validator tools in Node.js or other ecosystems where
[00:07:08]
you can actually verify before you consume the, the data, the request data, you can
[00:07:16]
verify that it conforms to that specification.
[00:07:20]
Which is also what made my job easy is I used Elm JSON schema packages that already existed.
[00:07:26]
I got to build on others work, just like open APIs building on others work.
[00:07:30]
Oh, cool.
[00:07:31]
What did, what did those tools help you do?
[00:07:35]
Instead of having to write a parser for everything, I only had to write a parser for or decoder
[00:07:42]
parser for just the open API portion of the schema.
[00:07:46]
So all like the body decoders and encoders and everything that's all handled by existing
[00:07:51]
Elm packages.
[00:07:53]
So it probably saved me, I would guess a few, few good months at least.
[00:07:59]
That makes sense.
[00:08:00]
So what are some popular APIs out there that, that use open API?
[00:08:05]
Well, I don't know how popular it is.
[00:08:07]
My start was actually with Square.
[00:08:10]
When I was working at Square some years ago, they produced an open API schema through a
[00:08:16]
series.
[00:08:17]
There's, there's was not handwritten.
[00:08:18]
There's was actually generated.
[00:08:19]
I believe it was proto buffs.
[00:08:22]
They used to generate the open API schema and then that was both available to customers
[00:08:28]
to use as from a development standpoint, as well as through various other things.
[00:08:34]
So that's where I got my start.
[00:08:36]
They generated an open API specification from proto buff.
[00:08:40]
I believe that's what it was.
[00:08:42]
If I remember right, yes, various teams write their own proto buffs.
[00:08:47]
Those are merged into one giant proto buff essentially, which then generates an open
[00:08:53]
API schema.
[00:08:55]
Is it like a proto buff specification?
[00:08:57]
Okay.
[00:08:58]
Okay.
[00:08:59]
Cause I have never used proto buff.
[00:09:01]
The only reason I know about proto buff is about, is because Evan talks about proto
[00:09:06]
buff in relation to Jason and all that.
[00:09:09]
And was like, Oh, okay.
[00:09:11]
Proto buff is a, is like Jason with different semantics.
[00:09:15]
It's like, so how do you make a specification from Jason?
[00:09:20]
Which, okay.
[00:09:21]
You get, you have a Jason specification of the, sorry, as a open API specification.
[00:09:25]
But okay.
[00:09:26]
You get what I mean.
[00:09:27]
Yeah.
[00:09:28]
Yeah.
[00:09:29]
It's just a series of specification transformers.
[00:09:31]
Essentially you're going from one specification to another format to another format and eventually
[00:09:37]
you get SDKs and can write code and yeah, send data.
[00:09:40]
But yeah.
[00:09:42]
So that was, that was my first experience.
[00:09:45]
I had seen it before them, but that was my first like real hands on Leonardo.
[00:09:51]
Most people in the, in the Elm community know him as mini bill.
[00:09:54]
He actually helped me a ton with the package and both the Elm package and the CLI tool
[00:10:01]
ton to ton.
[00:10:03]
He was using it for Spotify.
[00:10:06]
So they, they have their own open API spec.
[00:10:10]
He was using it.
[00:10:11]
I have never actually seen what he built with it.
[00:10:12]
I know he was building some Spotify tool for himself though.
[00:10:16]
So and it works for him, which is awesome.
[00:10:18]
That's the whole reason I built this for, for people to be able to use it.
[00:10:23]
Stripe I know from my time at square stripe being a competitive squares, like they have
[00:10:27]
their own open API spec that people can use.
[00:10:31]
I know there are lots of others.
[00:10:33]
Most, I feel like most companies these days have some type of open API spec.
[00:10:37]
I remember at my, at one of my previous companies, we, we had an open API spec or swagger spec
[00:10:44]
because that's what it was called back then, right?
[00:10:48]
And yeah, it was only for internal use, but that helped us make sure that the back end
[00:10:52]
and the front end were in sync and having the spec also allowed for automated tests.
[00:10:58]
So that was quite nice.
[00:11:00]
Did you build any back end with those specific, with those specifications as well?
[00:11:07]
Or I have not.
[00:11:09]
I have been dreaming that somebody will use it on something like Lambda era for their
[00:11:14]
back end, because a lot of these times these SDKs are more designed for back end because
[00:11:20]
you're holding API keys and other secret keys that you don't want to expose to the client,
[00:11:26]
especially payments.
[00:11:28]
Payments is a great example.
[00:11:29]
If you were connecting up to Stripe, you're given an API, a secret API key that is just
[00:11:34]
for your company.
[00:11:35]
You don't want to expose that to your customers to the front end.
[00:11:40]
Yep.
[00:11:41]
So you put that, you know, in your Lambda era back end and can do your communication with
[00:11:47]
Stripe from Lambda era back end and present some type of interface to the front end.
[00:11:52]
And yeah, there you go.
[00:11:54]
Yes.
[00:11:55]
So I don't have the business ideas yet to implement that and use that, but I know it's, I know
[00:12:01]
there's something there.
[00:12:02]
Yeah.
[00:12:03]
That's very cool.
[00:12:04]
I've heard too that the ChatGPT plugins are built around open API specifications, which
[00:12:12]
is really confusing because the company is open AI, but they use open API, but they're
[00:12:17]
not related.
[00:12:18]
And also there's nothing open about open AI anymore.
[00:12:22]
All that aside, it's...
[00:12:24]
I've also been confused about reading open AI and...
[00:12:29]
Yeah.
[00:12:30]
Like in my mind it's like open API because I was researching this episode or the other
[00:12:35]
way around.
[00:12:37]
Gets me every time.
[00:12:39]
And yet they actually are related because ChatGPT uses open API specifications to basically
[00:12:45]
build a plugin.
[00:12:47]
You give it a specification that tells it how to interact with your API and then use
[00:12:54]
plain English to describe how to use those endpoints.
[00:12:59]
So can you generate an open API spec using open AI?
[00:13:03]
You totally can and it's really good at that too.
[00:13:06]
Oh.
[00:13:07]
Yeah.
[00:13:08]
Full circle?
[00:13:09]
Short circle, but...
[00:13:10]
See, could use open AI to talk to Lambda era back end.
[00:13:15]
Yeah.
[00:13:16]
Yes.
[00:13:17]
Exactly.
[00:13:18]
I like it.
[00:13:19]
Wait, does Lambda have endpoints, back end endpoints?
[00:13:23]
I believe...
[00:13:25]
I don't know if they're still in the labs testing area or if they're released fully or not,
[00:13:33]
but it does have a way to communicate through REST to Lambda era back end.
[00:13:37]
Okay.
[00:13:38]
I missed that.
[00:13:39]
Yeah.
[00:13:40]
Interesting.
[00:13:41]
Makes me wonder now, could you use open AI or co-pilot or something to look at your
[00:13:47]
Lambda era back end code to generate an open API spec that you could then feed back into
[00:13:53]
open AI to communicate to Lambda era in production?
[00:13:57]
Snake eating his tail.
[00:14:02]
We are in the inception part of the episode.
[00:14:06]
Yes.
[00:14:07]
It's fun though.
[00:14:10]
I don't know.
[00:14:11]
I like the code gen side of things.
[00:14:14]
With open API makes for a lot of, I won't even say code, just generation of information,
[00:14:22]
open API.
[00:14:23]
Makes it really, already messing it up.
[00:14:26]
Makes it really easy.
[00:14:28]
Just go to Swagger.
[00:14:29]
Swagger.
[00:14:30]
Swagger, yeah.
[00:14:31]
Swagger is...
[00:14:32]
So yeah, those who aren't familiar with open API or only familiar with Swagger or confused
[00:14:38]
by it all.
[00:14:39]
Swagger is open API version two or lower.
[00:14:44]
I don't know the history there, but they changed names between version two and three.
[00:14:50]
They didn't have enough swag.
[00:14:52]
They lost all their Swagger when they became open.
[00:14:54]
But they were open about changing the name.
[00:14:58]
So you mentioned, yeah, that it was for V2 and below.
[00:15:03]
And when you open the Elm package you wrote, it says here are all the features we support
[00:15:09]
from V3 and V3.1.
[00:15:13]
And V2 is in the works, and maybe V3 is in the works as well.
[00:15:17]
I don't remember.
[00:15:18]
Kind of.
[00:15:19]
When I have time, I work on it.
[00:15:21]
Yeah.
[00:15:22]
So 3.0 points.
[00:15:25]
Why?
[00:15:26]
I don't even know why there's a why there.
[00:15:29]
Is not supported yet.
[00:15:30]
3.1 is mostly.
[00:15:33]
What is hard about supporting that or is it just like it needs to put in some work or
[00:15:39]
is it like a very different system?
[00:15:43]
Not very different.
[00:15:44]
No, just time to go back through their docs and see what changed between the two versions.
[00:15:51]
They are mostly compatible versions.
[00:15:53]
I haven't heard of anyone with like 3.0 not being able to use it.
[00:16:00]
I mean if you go back to Swagger, any of the two 2.x versions, you might have to do a few
[00:16:06]
tiny changes, but they're mostly compatible.
[00:16:10]
So there's the CLI and there's the Elm package.
[00:16:13]
The CLI is there to generate Elm code for people to use the SDK.
[00:16:20]
Or I guess it is the SDK then.
[00:16:23]
Or yeah, to use the rest, the rest endpoints.
[00:16:27]
What about the Elm package?
[00:16:29]
Is that meant to be used by people or is it just like you built the parser for the open
[00:16:37]
API spec and you thought it might be useful for other people as well?
[00:16:42]
Should I ever look at this package is what I meant?
[00:16:45]
Yes.
[00:16:46]
I do think there are benefits to it.
[00:16:50]
So going back to Square, all this came to me as something I wanted to work on when I
[00:16:57]
was at Square because of how we used it.
[00:17:01]
So I mentioned earlier that we would generate this open API spec.
[00:17:07]
But what happens from it there?
[00:17:08]
We didn't just give it to users.
[00:17:11]
One of the things we would do with it was send it off to another company called API
[00:17:15]
Matic to generate SDKs.
[00:17:19]
So that's kind of where I was like, oh, that's really cool, but they don't make an Elm SDK.
[00:17:23]
I want an Elm SDK.
[00:17:25]
I'm not going to pay them to make one.
[00:17:26]
I'll just do it myself.
[00:17:28]
So it's like, all right, well, in order to do that, I need to be able to parse the spec.
[00:17:32]
So that's kind of where the package started.
[00:17:36]
And then in addition to that, we would also take the spec and we would generate documentation.
[00:17:42]
So if you go to Square's developer docs, a significant portion of that is run through
[00:17:50]
the spec.
[00:17:51]
We take the spec and they would add markdown and other documentation into the spec written
[00:17:58]
by tech writers.
[00:18:01]
And you have a website, basically, then from that.
[00:18:05]
So if you wanted to, if your company exposed an open API spec that other people could use,
[00:18:13]
you could, in addition to that, generate all of your documentation from that same spec,
[00:18:19]
which is quite handy, then your documentation always matches your SDK, always matches your
[00:18:24]
backend.
[00:18:25]
Yeah.
[00:18:26]
It seems like a, I mean, if you're using REST endpoints, it seems like a great way to go
[00:18:30]
to get a lot of the tooling benefits that GraphQL would provide.
[00:18:35]
It seems like a great alternative to GraphQL.
[00:18:38]
And of course, GraphQL has its share of tradeoffs as well.
[00:18:44]
The whole overfetching, GraphQL is trying to solve overfetching and N plus one database,
[00:18:53]
you know, N plus one queries from the front end, but it makes it really hard to avoid
[00:19:00]
N plus one queries in the database because you have to tack on all these additional queries
[00:19:06]
and use creative ways to like aggregate them into one sort of query runner, which is very
[00:19:13]
difficult is a difficult problem.
[00:19:15]
So, so some people like using REST APIs for their back end.
[00:19:18]
And if you do, why not use open API and if you get a nice specification, it's way nicer
[00:19:24]
to use from, from Elm.
[00:19:26]
Also not everything goes through a get request with, with GraphQL, everything goes through
[00:19:32]
get request, right?
[00:19:34]
Or post or post request.
[00:19:36]
Yeah.
[00:19:37]
Oh, post request.
[00:19:38]
Was that it?
[00:19:39]
Yeah.
[00:19:40]
Yeah, I think it's two either.
[00:19:41]
But yeah, usually people do post request with a body.
[00:19:44]
Wait, you can do either.
[00:19:45]
Can you also do it with a delete request?
[00:19:48]
Maybe.
[00:19:49]
If you're a psychopath, then yes.
[00:19:54]
Yeah.
[00:19:55]
Yeah.
[00:19:56]
No, I think the comparisons to the GraphQL, I think are, are very warranted.
[00:20:03]
It is very, very similar in many ways to, to working with GraphQL.
[00:20:07]
You get a lot of similar tooling, a lot of similar typed benefits in a way, a lot of
[00:20:13]
similar guarantees.
[00:20:16]
I think I haven't worked with GraphQL long enough.
[00:20:20]
I've worked with REST for over a decade now.
[00:20:22]
I've worked with GraphQL for over a year.
[00:20:25]
So I feel like I still have a lot of learning on how to best use GraphQL.
[00:20:30]
I think with REST, it's a little bit more fixed in terms of what to expect, which is
[00:20:36]
kind of nice.
[00:20:38]
There's also, so other, other ways, since we were talking about, would, would you rune
[00:20:45]
use, use the Elm package?
[00:20:47]
Another thing I've been wanting to explore and bring it back to beginning with Martins.
[00:20:53]
I know there is a Martin already exploring this.
[00:20:56]
Which one?
[00:20:58]
Which one?
[00:21:00]
Oh, I'm trying to, I'm blanking on his last name.
[00:21:04]
Stuart?
[00:21:05]
No, no, no.
[00:21:07]
Wait, another one?
[00:21:09]
Yes.
[00:21:11]
We were talking in Elm, the Elm online meetup a few weeks ago about OpenAPI because the
[00:21:21]
company works for is using it.
[00:21:23]
They're actually using the package, not the CLI tool.
[00:21:26]
So you can too.
[00:21:28]
But they're looking at doing form generation.
[00:21:33]
So because it's using JSON schema under the hood for the bodies of the posts and gets
[00:21:41]
and deletes, you could just use whatever else exists for JSON schema and you could do form
[00:21:50]
generation.
[00:21:51]
I think he is actually using Dillon, your library as well, for the form side.
[00:21:56]
So he's kind of combining the Elm Open API and Elm forms to do form generation.
[00:22:03]
All of these schemas definitely, you squint your eyes and you're like, hmm, these do look
[00:22:10]
similar.
[00:22:11]
Yeah, in the blog post you wrote about effortless SDKs, which introduces Elm Open API.
[00:22:19]
You do talk about forms.
[00:22:20]
So is this something that people can do already or is this something that you want to work
[00:22:24]
on later?
[00:22:27]
Something I want to work on later.
[00:22:28]
When I was at Strange Loop, I was it was chatting with someone who they know they're Martin.
[00:22:35]
No, no.
[00:22:37]
I want to say if my memory is not failing me, I believe his name was Kevin.
[00:22:43]
He was doing some biomedical stuff prior and doing some form generation and had been using
[00:22:51]
ClosureScript for the form generation and was just trying to explore what else was out
[00:22:57]
there.
[00:22:58]
His back end was in Rust.
[00:23:00]
That's how the machines were communicating with the web interface.
[00:23:04]
And so he was exploring what else is out there.
[00:23:06]
I was like, you know, I think there is something Elm could do there.
[00:23:10]
I don't know for certain, but I think there is something.
[00:23:13]
We got to chatting about JSON schemas, which does most of the data portion.
[00:23:20]
And then there's also UI schema, which is UI definitions for rows, columns, and a bit
[00:23:28]
of other stuff to use in conjunction with JSON schema for defining forms.
[00:23:34]
So there is there is an existing schema for defining forms on top of JSON schema.
[00:23:40]
So I think I think I could probably my my idea is to try out something in that area.
[00:23:46]
I don't know what it looks like yet.
[00:23:48]
I haven't haven't had the time to get around to it.
[00:23:51]
But I think there could be something there.
[00:23:54]
So a little bit confused.
[00:23:56]
Can you define all the necessary validations on data through the open API spec?
[00:24:03]
Or is it too limited?
[00:24:05]
I'm guessing you can do relationships between two objects or whatever.
[00:24:09]
But relationships are not certain.
[00:24:12]
You might be able to.
[00:24:13]
I'm not 100% certain offhand.
[00:24:15]
You can define so it's not on the open API that's on.
[00:24:18]
So the validation is done on JSON schema, which is confusing.
[00:24:23]
But yeah, but open API spec is JSON schema also.
[00:24:29]
It uses JSON schema for defining request bodies and response bodies.
[00:24:36]
OK, because you did mention like, yeah, you can say that a price has to be a minimum zero.
[00:24:43]
So there are some validations.
[00:24:45]
So I'm curious about like, if I'm using Elm open API, so I've I've run the CLI tool.
[00:24:56]
I've generated some code for the Spotify API or some open API spec.
[00:25:02]
And now I've got a folder full of generated Elm code that I can use for all of the rest
[00:25:10]
endpoints in my specification.
[00:25:12]
Tell us a little bit about that code.
[00:25:14]
Is it like what are the what are the functions that it's giving giving you to consume these
[00:25:19]
APIs?
[00:25:20]
What do those look like?
[00:25:21]
Yeah, it is it is giving you your standard HTTP command requests.
[00:25:29]
Say you had a create product definition endpoint that so you would give you a create product
[00:25:38]
definition, it would take whatever data is necessary for actually creating a product
[00:25:44]
name price, something along maybe a description, optional description.
[00:25:50]
It would also provide you all the types for what a product is encoders decoders if you
[00:25:55]
need to use it somewhere else.
[00:25:58]
It will auto wire up all the encoding decoding for you in the request itself.
[00:26:03]
But if you need to use it to store in local storage or some something else outside of
[00:26:10]
the API, all that is provided for you are exposed for you.
[00:26:14]
That is a pretty cool detail because we just talked about concurrent task in last episode.
[00:26:22]
And it defines its own tasks, meaning that you could probably not reuse the HTTP request
[00:26:29]
if you wanted to be able to use them concurrently.
[00:26:32]
But if you have the building blocks to do that manually, you could still do it, although
[00:26:37]
you probably still would like to have some kind of cogeneration because it's a lot of
[00:26:42]
code otherwise.
[00:26:43]
Don't recall from the episode or from reading the docs.
[00:26:47]
If you can mix Elm tasks with Elm concurrent tasks.
[00:26:51]
No, no, not really.
[00:26:53]
I mean, it's its own thing.
[00:26:55]
You definitely can't create an Elm concurrent task from an Elm task.
[00:27:01]
Okay, okay.
[00:27:02]
Yeah, so there would be a limitation there because I do I do also expose a task version.
[00:27:08]
So there is like a there would be a create product, which we're going to turn a command.
[00:27:12]
And then there's also, I don't remember the naming scheme I use, but it is like create
[00:27:17]
product task that would create a task as well.
[00:27:21]
Just in case you don't I don't know what what you're going to want to consume.
[00:27:25]
Maybe you need both.
[00:27:26]
Right.
[00:27:27]
So you have the create article, create article task, or you have the decoder article encoder,
[00:27:36]
you have everything is generated and exposed.
[00:27:41]
That's nice.
[00:27:42]
The one thing that is not not released yet, I am working on it.
[00:27:47]
I think I'm getting close to something that I like is don't make promises on this podcast.
[00:27:53]
Don't.
[00:27:55]
No, no, I am actually decently close, but is is is air handling instead of returning
[00:28:02]
just an HTTP error.
[00:28:05]
The the spec or open API specs can also define what type of error to return what the error
[00:28:13]
messages can look like, which I think is very useful.
[00:28:20]
That's that is one thing that is.
[00:28:21]
Yeah, that's really nice to have.
[00:28:23]
So I'm trying to come up with a better air type that is specific to your API so that
[00:28:31]
you get this custom air type.
[00:28:34]
And then if it can't if for whatever reason you're back and returns something invalid
[00:28:39]
or some other network traffic error occurs that you can fall back to to a regular error
[00:28:47]
of some kind.
[00:28:48]
So that's that is the my experiments with that are a little weak.
[00:28:52]
I've I've been using the real world app for a lot of my local testing and local experimentation.
[00:29:00]
And that only defines two custom errors, which are there's a 401 on authorized error.
[00:29:06]
And there is another one called generic error in the spec I'm looking at, which is the error
[00:29:11]
for everything else.
[00:29:13]
Yeah, so and I've I've also learned from this that there are actually multiple implementations
[00:29:19]
of the real world backend in terms of what that spec looks like, which means sometimes
[00:29:25]
my test work or sometimes my experiments work with the backend that I'm hitting and sometimes
[00:29:29]
they don't.
[00:29:30]
So that's that's my latest or my current current work on the open API stuff.
[00:29:36]
Yeah, it it's very cool how if a if an endpoint requires an authorization token or a particular
[00:29:45]
query param with a given name, the code that elm open API generates shows reflects that
[00:29:51]
so it gives you the record argument to call the function has the authorization token and
[00:30:01]
the query programs that you need to pass in of the correct types.
[00:30:05]
One thing I can't help but think about.
[00:30:07]
And if you're playing Elm Radio bingo, now's the time to get out your bingo cards.
[00:30:12]
What if you wanted to create some sort of opaque type, let's say,
[00:30:16]
Ding, ding, ding, ding, ding, please.
[00:30:21]
I think that was going to be my next question as well.
[00:30:27]
What a surprise.
[00:30:28]
Yeah, so like with with Done Kerns on GraphQL, for example, you know, I'm I'm a big fan of
[00:30:36]
using opaque types as much as possible, both decoding into and using, you know, custom
[00:30:43]
input types and that sort of thing, like for an authorization token, for example, I might
[00:30:48]
want to have a wrapped opaque type that is actually an authorization token and maybe
[00:30:55]
that I can't accidentally pass a string oops, that I mixed up the names of the authorization
[00:31:02]
token and the username and now that's getting logged to, you know, the console or something
[00:31:08]
like that, right?
[00:31:10]
And same with like getting back data, things like enums or things like a non negative integer,
[00:31:16]
like a price, I might want to have an opaque type to make sure I don't cross wires for
[00:31:22]
these different result types that I'm getting back and have certain guarantees that I that
[00:31:28]
I know invariance about those values.
[00:31:31]
Has that been on your radar at all?
[00:31:33]
Have you thought about that?
[00:31:34]
Is that compatible with this approach?
[00:31:36]
Yes, yes, yes.
[00:31:38]
A thousand times yes.
[00:31:41]
I'm glad you didn't put a no in there because I didn't know to which question it would
[00:31:47]
have been an answer to.
[00:31:49]
I honestly don't know which one I would have lined up with.
[00:31:54]
So the the the tokens that had not specifically crossed my mind, like that specifically, I
[00:32:00]
do really like that, though.
[00:32:02]
I don't know if I could make a special exception just for that or not.
[00:32:06]
I'm tempted to.
[00:32:08]
I do.
[00:32:09]
So my thinking has been that in this kind of goes with the form generation as well, because
[00:32:17]
I think it kind of benefits both sides is the schemas.
[00:32:21]
All these schemas have a way to define your own properties on the schema.
[00:32:26]
You usually form like X dash custom property name and that can do whatever you want.
[00:32:33]
Essentially, it can mean whatever you want.
[00:32:36]
And I think there might be a way to use that to define custom ways to handle encoding and
[00:32:45]
decoding and maybe form, form handling as well.
[00:32:50]
So like an example, there would be units.
[00:32:56]
No one's familiar.
[00:32:57]
There there is a great package from Ian McKenzie called Elm units.
[00:33:01]
That is really fantastic for handling anything with any type of unit value, like feet to
[00:33:08]
meters, you don't want to mess that up.
[00:33:11]
So why not use a package that just does it for you?
[00:33:14]
Similarly, if you're dealing with endpoints that are to have that kind of data similar
[00:33:19]
to a token, like I don't want to pass in the wrong string for a token, I don't want to
[00:33:23]
pass in feet when I want to use meters in my endpoint.
[00:33:27]
That would that could be catastrophic in some cases.
[00:33:32]
There are spaceships that have blown up because of that.
[00:33:35]
So I haven't had the chance to experiment yet in code itself, but I've been thinking
[00:33:41]
for months now that there is probably a way to handle automatically pulling in a package.
[00:33:49]
So a way to find I want to use this package for this type of data and have it auto encode,
[00:33:56]
decode that type of data, wrap that type of data in whatever wrapper it needs.
[00:34:02]
Yeah, I really, I think there's a lot of potential in that area, a lot of potential.
[00:34:07]
That would be, that would be amazing.
[00:34:08]
Yeah.
[00:34:09]
So like in the, in the case of Dillon Kern's Elm GraphQL, what I did is I introduced, you
[00:34:14]
know, this custom scalar codecs file where you, for each custom scalar in your schema,
[00:34:23]
you define a codec encoder and decoder pair.
[00:34:29]
And now the nice thing about the GraphQL spec in that regard is you do have this concept
[00:34:36]
of a nominal type, a type which is not just a set of fields of particular types.
[00:34:44]
It's like, this is a, you know, a unit of, of feet or of meters, like you can specify
[00:34:53]
custom scalars and give it a name and say, well, the underlying representation might
[00:34:59]
be exactly the same as this other thing, but it's not like a, if it quacks like a duck,
[00:35:03]
it's like this is units and meters, not to be confused with some other unit that we use,
[00:35:10]
unit and millimeters or something, you know.
[00:35:14]
So the GraphQL schema definition language has this notion of users being able to define,
[00:35:20]
to define their own custom scalars.
[00:35:23]
And then Dillon Kern's Elm GraphQL just provides a way to define encoder, decoder pairs for
[00:35:29]
each of those custom scalars in the schema.
[00:35:33]
Now with OpenAPI and JSON schema, I wonder like, is it more duck typed by its nature
[00:35:40]
in the sense of it's just saying, well, this is an integer and its minimum value is this
[00:35:47]
and it doesn't have a maximum value.
[00:35:49]
And you're not really naming things in one central location where you give something
[00:35:56]
a name and then you reference that named thing you defined at the top.
[00:36:01]
I will say it's limited by JSON a little bit.
[00:36:03]
So the defaults are things like a generic number or a string, an array, they're very limited
[00:36:13]
in that scope initially.
[00:36:15]
There is a portion of the schema that is just for defining, called objects for, since it
[00:36:24]
kind of fits with JSON a little bit.
[00:36:26]
So you, when you define your product, your product could be a ruler.
[00:36:33]
So a ruler measures things, but doesn't tell you just by the name if it's in imperial or metric,
[00:36:39]
you have to actually see it or have something else.
[00:36:42]
And that might actually be defined more in that object schema portion of the full schema,
[00:36:48]
the full spec.
[00:36:50]
So there you could, you might have a minimum of like, a ruler is never going to be less
[00:36:56]
than zero, so a min of zero and a max of maybe 50 or something, I don't know.
[00:37:02]
You, on top of that, if you wanted to define units, so if you wanted to know if it was
[00:37:07]
imperial or metric, you would most likely, I think, use a custom property that was unique
[00:37:13]
to your API.
[00:37:15]
Those, those are quite common using custom properties.
[00:37:19]
It's one thing that is probably the one thing I'm not sure entirely how to support yet in,
[00:37:25]
in my package or my CLI tools, because they, they can be anything almost that
[00:37:31]
basically any JSON schema definition can be used there.
[00:37:35]
Although I do like that you mentioned having a separate config specifically for some of this.
[00:37:41]
Think I hadn't considered that entirely, but I kind of like that because if I'm using,
[00:37:46]
if I'm using Spotify's API, I can't modify their, their API spec, their open API spec.
[00:37:53]
I can't modify it's not my, I mean, I could download it and edit it.
[00:37:57]
But then when they release a new version, I have to go and manually edit it again.
[00:38:01]
And that's, that's not fun for somebody.
[00:38:04]
So having the option to either include your own custom properties or for a third party
[00:38:12]
spec, being able to define your own local config to add on to it, essentially that
[00:38:18]
that could be a very nice route to go.
[00:38:21]
Yeah, if there was some way to hook into that, that would, because I guess what often happens
[00:38:25]
is like things, you know, tools around GraphQL and JSON schemas, you know, a lot of people
[00:38:32]
are using them with TypeScript maybe, and they're just like, okay, let's just add TypeScript types.
[00:38:40]
And they use sort of the lowest common denominator, primitive types to describe it.
[00:38:45]
And that, and that's pretty nice.
[00:38:46]
And then with, you know, with TypeScript, you have an enum and it can be five different strings.
[00:38:51]
And then in TypeScript, you can just describe that.
[00:38:53]
You can say, this is one of these five strings, which is really nice.
[00:38:57]
But at the same time, it is, you know, we also like opaque types and being able to have certain
[00:39:04]
guarantees around those things.
[00:39:06]
So we also like custom types.
[00:39:08]
And so if you want to do more nominally typed things, TypeScript isn't, doesn't give you the same
[00:39:15]
guarantees with that.
[00:39:16]
And it's not the path of least resistance.
[00:39:18]
And a lot of tools end up just catering to that sort of, hey, let's nicely describe what
[00:39:23]
primitives we're using everywhere.
[00:39:25]
But to me, the really interesting thing is when you go beyond that and you say like, okay,
[00:39:29]
well, it's giving us all these primitives as a starting point, but then we can actually
[00:39:34]
describe, give more semantic meaning to that set of primitives and use opaque types around them.
[00:39:40]
Yeah, I do.
[00:39:43]
There absolutely is a reason for, like you pointed out, being a being very minimal in your
[00:39:49]
definitions.
[00:39:50]
We ran into that a lot at Square with our open API spec in that we were having to support, we
[00:39:55]
support TypeScript, which was usually fine.
[00:39:58]
Ruby was usually fine.
[00:40:01]
Java and C sharp were always a bit frustrating, more, more due to like the generated definitions
[00:40:07]
would, you know, the, if everyone's familiar with working with, especially with Java, you would
[00:40:12]
end up with function calls that have like six nullable arguments.
[00:40:18]
And so sometimes you just have to pass in null like four, five, six times.
[00:40:24]
That was always fun.
[00:40:26]
But yeah, so you don't know, like you're having to potentially target many different languages
[00:40:32]
with very different semantics.
[00:40:34]
And it's hard to define something that fits all of them equally.
[00:40:40]
Everyone's going to be slightly disappointed in their, in their generated SDK.
[00:40:45]
So if you, yeah, if you have your own little config, you say, cool, I like this, but I'm
[00:40:51]
going to tweak it to fit my language or even to fit my app, like just to be able to fit
[00:40:57]
your specific project.
[00:40:58]
Very handy.
[00:41:00]
Very nice to have too, too many ideas now.
[00:41:03]
Yeah.
[00:41:04]
To throw another idea out there.
[00:41:06]
It, uh, like, I, well, I'm not sure if you do this at all.
[00:41:11]
I think maybe you don't, but for enums in a JSON schema, you can describe like an enum
[00:41:17]
like these are the five possible string types for this value.
[00:41:20]
Right.
[00:41:21]
Do, do those get generated as string values or custom types?
[00:41:27]
Believe they get generated as custom types right now.
[00:41:30]
They do.
[00:41:31]
It was, they do.
[00:41:34]
I'm fairly certain they do.
[00:41:36]
I went back and forth on that a lot actually.
[00:41:39]
Intuitively, you think that's the right way to go.
[00:41:42]
However, there are, there are weird edge cases, um, which I only know from dealing
[00:41:48]
with them at square in that people like to sit on one version of an SDK for a very
[00:41:53]
long time, sometimes.
[00:41:55]
And so if you, you keep updating your backend to define new, new variations on
[00:42:03]
that enum, the SDK might not be able to handle that anymore.
[00:42:07]
Uh, and so we would run into issues with customers on very old versions of the SDK,
[00:42:13]
not having the time, maybe even to upgrade.
[00:42:15]
Maybe they're just swamped with other feature work.
[00:42:17]
They can't, you know, spend a day or a week even upgrading the SDK, but you
[00:42:24]
don't want to break them either.
[00:42:26]
So we ended up, at some point, we ended up switching most of our SDKs back to
[00:42:31]
using strings for enums for that specific reason, which is a bit disappointing, but
[00:42:37]
it also kind of makes sense from that standpoint.
[00:42:40]
I do think this did inspire me though.
[00:42:43]
So maybe that's where opaque types come in.
[00:42:47]
You could probably with Elm instead of exposing the strings directly.
[00:42:52]
Right.
[00:42:53]
Yes.
[00:42:54]
I wonder if there's some way to do something opaque or just expose them through.
[00:42:59]
Like if you want to create them, you just expose a function name that behind the
[00:43:03]
scenes is still generating a string, but hidden within some opaque type.
[00:43:07]
Yes.
[00:43:08]
So you could totally do that.
[00:43:10]
And, and the benefit to that would be that, uh, from an API point of view, of
[00:43:16]
course, if you're not actually publishing a package with this generated SDK from
[00:43:23]
Elm Open API, then the, the breaking changes thing is more of a, you know, you
[00:43:28]
know, when it's broken because your code doesn't compile, but, but from the point
[00:43:32]
of view of like publishing breaking changes, you can add an enum variant and it's
[00:43:39]
not a breaking change with an opaque type because you're exposing a new function.
[00:43:43]
So that's a minor version bump, not a major version bump.
[00:43:47]
But if you remove one, then it's a major breaking change, uh, because, because it
[00:43:53]
is, because it could break someone's code.
[00:43:55]
Uh, whereas with a non opaque custom type, you're in, we should, uh, we should lobby
[00:44:00]
to, to call them explicitly non opaque.
[00:44:03]
We should lobby to call them non opaque types.
[00:44:08]
Transparent types.
[00:44:11]
Bad types.
[00:44:12]
Yeah.
[00:44:12]
Bad types, the good types and bad types.
[00:44:14]
It is considered in a published API, a breaking change.
[00:44:18]
If you, um, if you add a variant, because, you know, you could have a case
[00:44:22]
expression that you now have to handle that one additional thing.
[00:44:25]
Now, on the other hand, so, so I'm able to do case expressions is handy.
[00:44:30]
So that's a trade off too.
[00:44:31]
Yeah.
[00:44:31]
But, uh, you mentioned if you remove, uh, a possibility in the API, then that is
[00:44:38]
also a breaking change for the backend, right?
[00:44:41]
Because you're not, not, you're now not able to call it to call, get
[00:44:46]
article with type, uh, some vegetable.
[00:44:50]
I don't know, like vegetables are not sold anymore.
[00:44:53]
Vegetables don't exist anymore.
[00:44:55]
Well, now you can't send that anymore.
[00:44:57]
And that's a breaking change also.
[00:44:59]
So that shouldn't happen in that case.
[00:45:01]
Usually the people go to a V two or V three of that same endpoint, right?
[00:45:08]
So usually what would happen is you would add a new, new, new vegetable type.
[00:45:15]
Uh, so carrots for whatever reason, you forgot to add carrots in, in your
[00:45:19]
initial release of your backend and you go and you're like, shoot, I forgot those.
[00:45:23]
You add carrots.
[00:45:24]
That's, that's stupid because that's the best vegetable.
[00:45:27]
I, I is a fantastic vegetable.
[00:45:29]
It's probably up in my top three.
[00:45:34]
But hey, you know, you were busy.
[00:45:36]
You, you were working 12 hour days for whatever reason to get this shipped.
[00:45:40]
Cause you're full and work.
[00:45:41]
Carrots before hands.
[00:45:43]
Yup.
[00:45:43]
Exactly.
[00:45:44]
So you ship it, a customer goes and installs version 1.0.0 of your SDK and they
[00:45:52]
go and they use it and they're like, great.
[00:45:54]
And they don't actually use that vegetable type.
[00:45:56]
They don't care about it, but it's still part of the response.
[00:46:00]
So the response decoder has to decode it.
[00:46:03]
They don't care about it where they don't care it.
[00:46:07]
Sorry.
[00:46:10]
Nice.
[00:46:10]
Nice.
[00:46:11]
They don't care about it.
[00:46:13]
So they, you go, you add carrots.
[00:46:16]
It's a non-breaking change.
[00:46:18]
It is.
[00:46:18]
It's a minor, minor thing.
[00:46:19]
You've added a new type.
[00:46:21]
Their decoder doesn't support it.
[00:46:23]
So their decoder, if it got carrots would fail because it doesn't recognize it, but
[00:46:30]
they're not actually using it.
[00:46:31]
So now you've broken their, their, their SDK for no, no apparent reason.
[00:46:39]
But if you wrapped it in just a very small, slim, opaque type that took it as just
[00:46:45]
like unknown, unsupported type, they can keep working.
[00:46:49]
They're fine.
[00:46:50]
And everyone else who wants carrots, they still have access to carrots.
[00:46:54]
The end user might still be defining a custom type that, you know, they're
[00:47:01]
going to be adding carrots to and handling that in their own code, right?
[00:47:06]
So in a way it, it passes the buck downstream.
[00:47:10]
So it's, it's an, it's an inherently challenging question.
[00:47:14]
There's no easy magic bullet to it.
[00:47:17]
Yeah.
[00:47:17]
If a new possibility arises, like a carrot gets introduced by the back end, then
[00:47:23]
your whole front end doesn't compile anymore because you've upgraded this back.
[00:47:28]
And now the front end needs to real quick handle that case.
[00:47:33]
I think there's, so there is an edge case here.
[00:47:36]
If you are working with open APIs internally within your company, odds are you
[00:47:42]
do want the custom type, regardless if it's opaque or not, you do want, you
[00:47:47]
don't want to use stringly typed enums.
[00:47:49]
You want the custom type because if, if your back end does change something, you
[00:47:54]
want that reflected in your front end.
[00:47:56]
However, if you are using this between companies, possibly between multiple
[00:48:02]
companies like, like Spotify or something like that, then the custom type isn't, I
[00:48:08]
think there's more risk involved with the custom type in terms of breaking the
[00:48:13]
customers, the customer's application, which you don't want to do.
[00:48:17]
There is still the edge case that they could forget to or customly handle
[00:48:22]
carrots or rutabagas or whatever else they decide they need to support or you
[00:48:27]
forgot to support.
[00:48:29]
But that's like one of the worst vegetables.
[00:48:34]
Oh, poor Vegas.
[00:48:37]
They deserve it.
[00:48:40]
But yeah, you don't want to, you don't want to break your customers and point, and
[00:48:44]
they might not have the time.
[00:48:45]
There isn't the same level of communication that you have between teams within a company.
[00:48:50]
So maybe that's something that is like a flag maybe on or in your config for when
[00:48:55]
you generate your own SDK from this.
[00:48:58]
Maybe I have something that says, I want custom types for enums or I want strings
[00:49:02]
for enums and let, let them choose or let them override to whatever fits their product
[00:49:10]
the best.
[00:49:11]
At work, we've had some, we use Elm GraphQL Dillon's version.
[00:49:16]
And some of the things are using custom types.
[00:49:20]
And that creates some problems.
[00:49:23]
For instance, when we do migrations, like the backend migrates to a new version where
[00:49:29]
it needs to support having older versions or newer versions of the front end talking
[00:49:34]
to it or the other way around.
[00:49:35]
Like you've seen Mario Rogers talk like all those variations.
[00:49:41]
Well, we need to support those.
[00:49:43]
And therefore we are thinking of like moving those custom types into stringified, stringified
[00:49:50]
types, which is not great.
[00:49:52]
But it is a lot more flexible.
[00:49:55]
It is a lot easier to ignore some things that are unknown.
[00:49:59]
And you just say, okay, well, the front end doesn't know about carrots.
[00:50:03]
But if it gets a carrot, what should, what should we do?
[00:50:07]
Should we just ignore the message or should we crash hard?
[00:50:13]
And in some cases, it's fine to just ignore it.
[00:50:17]
It really depends on the use case, obviously, or the situation.
[00:50:20]
So we also use GraphQL at work.
[00:50:23]
And I think the way we handle that is we generate a hash, essentially, of our GraphQL
[00:50:31]
schema or whole GraphQL schema and just upload that as a single file and then use
[00:50:38]
that as a versioning thing.
[00:50:39]
And if the front end sees that that has changed, then it will refresh the page, essentially.
[00:50:48]
Kind of like if you're on Slack on the browser or Discord or many other messaging apps, and
[00:50:54]
they're always like, a new version is available.
[00:50:55]
Click this button to update when it just reloads the page.
[00:50:59]
Or Discord, which will just not let you look at anything if it's outdated, which I get.
[00:51:09]
I totally get that.
[00:51:10]
That's fair.
[00:51:11]
Well, it really depends on the situation, right?
[00:51:14]
Because if you're on Discord, you're typing a message and it says, oh, I need to update.
[00:51:19]
Okay, well, let's refresh.
[00:51:21]
Your message is still in the box.
[00:51:24]
It's fine.
[00:51:26]
But if you're editing a very complex form, you don't want to lose your things.
[00:51:32]
If you're doing something that takes a long time to set up somehow, then you don't want to lose that.
[00:51:39]
Right?
[00:51:39]
I think it depends on the size of the company, too.
[00:51:41]
I don't know how big you're at CrowdStrike, right?
[00:51:44]
And I don't know how big CrowdStrike is, but if you get to the point where you have multiple
[00:51:49]
back-end teams interfacing with multiple front-end apps and one of them wants to change an enum
[00:51:55]
and add or remove that enum, that can be very difficult to coordinate across teams.
[00:52:01]
A lot more difficult than one back-end team and one front-end team, essentially.
[00:52:05]
Or like one front-end app and one back-end app.
[00:52:09]
Yeah.
[00:52:09]
So it mostly depends on who is your customer.
[00:52:12]
Is your customer the same mono repo application with one back-end, one front-end, and then they're
[00:52:19]
always in sync because there's always the same person who makes the back-end changes knows how
[00:52:26]
to do the front-end changes as well.
[00:52:28]
Or do you have back-end that is not always in sync with front-end or you use web components
[00:52:35]
or you're shipping part of your products in not the same way?
[00:52:39]
So yeah.
[00:52:40]
So the flexibility, right?
[00:52:43]
Yeah, it's the hard part.
[00:52:45]
I always like to view it as the hard part is communication.
[00:52:50]
And if the communication is hard, maybe strings are easier.
[00:52:54]
Communication is easy.
[00:52:55]
Then go with custom types when you can.
[00:52:57]
I mean, you communicate with chat GPT only with text.
[00:53:01]
Text.
[00:53:03]
Image is nowadays as well, right?
[00:53:05]
But yeah.
[00:53:05]
Yeah.
[00:53:05]
Yeah.
[00:53:06]
Yeah.
[00:53:07]
I was also thinking too, other things I've gotten to look at using this with, I know
[00:53:14]
super base is a back-end service like database slash back-end as a service.
[00:53:20]
You can also, I believe generate, I think they're swagger specs, but you can generate
[00:53:25]
those from it, which means you could build an L map on top of super base with generating
[00:53:32]
all the rest end points or the rest SDKs for your front-end.
[00:53:36]
I started experimenting with it, but yeah.
[00:53:39]
Again, have no business business ideas to actually build with it.
[00:53:43]
Is your Elm open API the first time that a production ready version has been shipped?
[00:53:51]
Like that.
[00:53:51]
I feel like there have been murmurings of this for a long time.
[00:53:55]
I mean, it's like an obvious fit for Elm because of its types.
[00:53:59]
There is a, if you're talking open API and Elm, specifically,
[00:54:06]
there is a Java based one.
[00:54:09]
Trying to remember, it was pointed out, I might have come across it a while ago,
[00:54:14]
but someone pointed it out to me recently.
[00:54:18]
That one, I think that one is, it's from open API, general or open API tools.
[00:54:24]
They release an open API generator and it can generate Elm.
[00:54:28]
There's also a swagger decoder Elm package, but not, it doesn't generate code as far
[00:54:36]
as I can sell.
[00:54:36]
Do you think everyone ever seen that at one point?
[00:54:39]
I probably did and then promptly forgot about it while looking at other packages.
[00:54:46]
The, so when the open API tools one got brought up to me, I was very curious to see
[00:54:52]
how ours differed because I didn't look at it at all while working on my stuff.
[00:54:57]
It is one thing I do like that it does is it splits out into multiple Elm modules,
[00:55:05]
kind of around types a little bit.
[00:55:08]
It splits it out based on, so it splits out your endpoints into one module, your
[00:55:14]
request body, response body objects into another file.
[00:55:18]
And I think there might be a third one it generates as well, which is kind of nice
[00:55:23]
because it breaks things up a bit.
[00:55:25]
I would like to do the same though, I think more around types themselves instead
[00:55:31]
of around functions.
[00:55:33]
I think that feels a little bit more Elm like to me.
[00:55:35]
I think the open API stool feels a little bit more Java to me.
[00:55:41]
And then same with, I noticed too, with how you call functions feels a little bit
[00:55:46]
more Java like and how they're written for it, bringing back a little bit to off
[00:55:54]
off stuff and tokens.
[00:55:57]
The API tools generation provide like a with authentication function to allow
[00:56:05]
you to add that bearer token or JWT or whatever it is to your request.
[00:56:11]
But it's always optional and you don't know if you need it or not for your
[00:56:14]
endpoint, which doesn't feel Elm like to me.
[00:56:18]
Like if this function requires, if this endpoint requires a token, then it's,
[00:56:24]
you should be forced to pass it in, which is what mine does.
[00:56:29]
If for every endpoint that requires a token, you're forced to pass in that token.
[00:56:32]
Otherwise, what's the point in requesting the data because it's just going to fail?
[00:56:37]
So I think, yeah.
[00:56:40]
So I think there's something there where maybe I definitely need to break mine
[00:56:44]
out so that you don't have an API module that's 5,000 lines long, 10,000 lines long.
[00:56:50]
How do you manage like staging API URLs versus dev URLs versus production URLs?
[00:56:59]
Is there a way to handle that?
[00:57:02]
I think there is.
[00:57:04]
I hadn't actually thought about it too much.
[00:57:06]
So the way the schema works is there is like a top level URL defined.
[00:57:13]
So Spotify's might be like HTTPS colon slash slash Spotify.com slash API.
[00:57:22]
That is then used for all the endpoints and the endpoints are then just slash
[00:57:27]
playlist or slash artists.
[00:57:31]
And it all gets joined together.
[00:57:34]
I believe you can define multiple top level API or yeah, top level URLs.
[00:57:41]
So I guess I'm not sure what that looks like, but that probably should be something
[00:57:45]
that's handled.
[00:57:48]
Yeah, because you could talk to like a real world backend.
[00:57:53]
But you can just say, well, whatever URL, whatever backend I'm targeting,
[00:57:58]
the URL is defined not at code generation time, but at runtime or right.
[00:58:04]
But that doesn't work with your approach as far as I can tell because you generate
[00:58:09]
that URL inside the you put that URL inside the generated code.
[00:58:14]
But that could be an argument rights as well.
[00:58:17]
Yeah, there there likely are a lot of additional arguments that need to be
[00:58:22]
optional ones, like completely optional.
[00:58:24]
The real world server might be behind a proxy of some kind or something like that.
[00:58:30]
And so being able to say with proxy URL or something along those lines would be
[00:58:36]
a very nice thing to prepend every URL requests that you make with that.
[00:58:41]
There could be other things as well, maybe cuss, maybe for whatever reason
[00:58:45]
you want to handle custom decoding just locally in your app.
[00:58:48]
You're like, the generated coding is great, but I want a little tweak on it.
[00:58:53]
Maybe in the special case, so like with decode map or something along those lines
[00:58:59]
might be might be a nice thing to have.
[00:59:01]
There are other ones too.
[00:59:02]
I know I want to add retry at some point.
[00:59:04]
Retry is something that's usually usually nice to have.
[00:59:08]
For request fails, maybe retry every 30 seconds incrementing by 20 seconds
[00:59:14]
or something along those lines for 15 times.
[00:59:17]
Wouldn't you be able to do that through the task API, the Elm task API anyway?
[00:59:22]
You could write your own retry.
[00:59:24]
Yeah, but it'd be nice to this is mostly a lot of these things are things
[00:59:28]
that I knew were request when I was working at Square from users
[00:59:33]
that were just really nice to have built in for the language.
[00:59:38]
So why make you write your own retry logic when I can just generate it for you?
[00:59:43]
Because it's never exactly what I want.
[00:59:48]
Yeah, I get the feeling too that you're talking Wolfgang about a sort of
[00:59:53]
pre-packaged SDK, something, the kind of thing that would be like NPM install
[01:00:01]
Spotify API or GitHub API or something.
[01:00:05]
And then it installs some JavaScript functions that let you use the API.
[01:00:10]
So you're talking about that sort of pre-packaged SDK, right?
[01:00:14]
Yes, yes.
[01:00:15]
Because that's, I find it very useful when I when I'm doing JavaScript
[01:00:21]
and I'm like, I need to talk to the service.
[01:00:24]
Oh, hey, there's a package I can talk to the service and it's no setup.
[01:00:28]
You there's a lot like you usually have to pass on a token once your API token
[01:00:33]
for that service once and then you get some type of pre-pre-setup request.
[01:00:39]
And then you just say, OK, go get me this data and you're done.
[01:00:43]
And then if you look at I'm trying to think of some but with those pre-packaged ones,
[01:00:50]
if you look at their REST APIs and you go to the rest docs,
[01:00:54]
it's it's a fair amount of work to get back to that same point.
[01:00:58]
And you're not sure, like, oh, shoot, did I type that wrong?
[01:01:01]
Did I type the URL correctly?
[01:01:03]
Did I remember to pass in the off token?
[01:01:07]
Did did they change their their timeout like or some limits?
[01:01:12]
Like maybe the API only allows you to make a request every 15 seconds.
[01:01:16]
Did I actually set that to 15 seconds or did I set it to 10 seconds
[01:01:20]
because I was looking at something else and got a number wrong.
[01:01:24]
So having those types of experiences is very nice.
[01:01:28]
And and I don't expect companies to go make Elm SDKs.
[01:01:34]
Like that's a lot of work to go and make SDKs for other languages.
[01:01:39]
So maybe I can help bring that experience, that nice quick.
[01:01:44]
I want to use a service with minimal effort, that experience to Elm.
[01:01:49]
Would you so let's say Spotify got big into Elm?
[01:01:53]
And they wanted to make it easy for people to use their SDK.
[01:01:59]
Would you recommend that they would generate an Elm package
[01:02:03]
or that they make their open API spec
[01:02:08]
readily available and give good instructions on how to generate
[01:02:14]
it's using your package, like especially with the idea of having
[01:02:19]
custom opaque types and all that.
[01:02:21]
Maybe it's better to have it to generate it yourself so you can edit
[01:02:26]
a bit instead of having a readily but fixed Elm package.
[01:02:33]
I think handwriting your own SDK for especially a larger API can be a lot of work.
[01:02:39]
Oh, no, in both cases, it would be using your your package,
[01:02:43]
but it would be publish publishing the results or telling people to
[01:02:48]
generate themselves using your tool.
[01:02:51]
I think it would be I think it'd be publishing would be my guess.
[01:02:56]
I think that if Spotify were really big into Elm,
[01:02:59]
my my thought would be you would go to their developer page.
[01:03:03]
They would have their first listed out their SDKs.
[01:03:07]
And those would be say then Elm and JavaScript and maybe Python.
[01:03:12]
And then they would after that list their API spec
[01:03:15]
and their rest endpoint documentation.
[01:03:18]
You'd probably still want the rest.
[01:03:20]
I mean, you'd still want the rest documentation regardless
[01:03:22]
because it's helpful for understanding the SDKs.
[01:03:26]
But you would list the SDKs for saying like, hey, we support these.
[01:03:30]
We put time into these because we want you to quickly get up to speed
[01:03:35]
and build an app with our stuff.
[01:03:37]
If you're using a language that doesn't isn't listed here.
[01:03:42]
Here's another way to go about an interface with our with our API.
[01:03:46]
Also, if I had one once I have some way to like
[01:03:50]
customize the generated SDK with it like extra config,
[01:03:54]
they could go into and like define their own extra stuff that they want.
[01:03:59]
Like, like, you know, a custom.
[01:04:01]
I'm trying to think what would be Spotify.
[01:04:03]
I don't know what Spotify I was going to say track length always
[01:04:06]
a positive number, but you can have tracks that are negative have negative time as well.
[01:04:11]
Oh, yeah, great way to hide hide audio in tracks is to go negative.
[01:04:15]
Yeah. And you can't if I remember.
[01:04:18]
So I know this from back to listening to CDs
[01:04:22]
because you can't really do that on a tape.
[01:04:23]
There's tape has a start and an end.
[01:04:25]
But in CDs, you have tracks that have a start and stop time.
[01:04:30]
If you have negative time, the only easy way to get there
[01:04:34]
is to finish the previous song and it will start the next song at the negative time stamp.
[01:04:40]
If you skip to the next song, it starts at zero.
[01:04:43]
And so you would completely miss that like hidden audio.
[01:04:46]
Oh, is that for instance, like to make the transition between songs nicer?
[01:04:51]
I don't know.
[01:04:52]
I maybe but maybe like a translate type thing.
[01:04:56]
Yeah. Oh, you can hide.
[01:04:58]
You could hide an entire song there.
[01:04:59]
I've had CDs back in the day where they had like negative four minutes
[01:05:04]
and would hide entire songs in a track.
[01:05:06]
What the hell? Yeah.
[01:05:08]
Oh, yeah.
[01:05:11]
I mean, there are songs where the play song, then they have like two minutes
[01:05:16]
of of nothing and then a little thing at the end.
[01:05:20]
Is that it as well?
[01:05:21]
Like and that's somehow good.
[01:05:23]
That's that's kind of.
[01:05:25]
But that's not the same because you can like you can easily tell.
[01:05:29]
Oh, my my track length is listed as six minutes and you get to four minutes.
[01:05:34]
And then it's just two minutes of white noise.
[01:05:36]
And you're like, why is it listed six minutes?
[01:05:38]
Let me like skip forward to it.
[01:05:40]
But if it's negative, that's not listed.
[01:05:43]
So the only way to really know it's there is to either like rewind.
[01:05:47]
Maybe I don't even know if that works.
[01:05:50]
I think you have to like finish the previous song and let it auto go to the next section.
[01:05:56]
So so if you're if you're a sucker for shuffling songs in an album,
[01:06:01]
like, no, you're not going to have it.
[01:06:03]
I knew shuffle as long as you finish the previous song.
[01:06:07]
But I don't know if that works in web with MP3s.
[01:06:09]
I don't know if MP3s can go negative.
[01:06:11]
It might just be a it might be a forgotten thing, a lost thing with CDs.
[01:06:16]
I'm very curious now.
[01:06:18]
I might have to after this, I'm going to look all this up now.
[01:06:20]
I'm really curious because I forgot.
[01:06:22]
You just play this.
[01:06:24]
Yeah. So yeah, no big type for that one.
[01:06:28]
No, in less in less than MP3s, you can't have negative time.
[01:06:32]
I don't know. Now I have to kind of hope it doesn't.
[01:06:38]
It is pretty cool to be able to do that.
[01:06:40]
So yeah, so they Spotify could go and add their own flavor
[01:06:45]
onto the SDK generation that really makes it shine.
[01:06:49]
That if you or I went and generated their their SDK ourselves, we wouldn't have.
[01:06:56]
So I know we've talked about it during one of many Elm Cogen episodes.
[01:07:02]
But it's still worth pointing out that it's really cool that in Elm,
[01:07:07]
when you generate these huge API files, you only get in a ship, whatever you use.
[01:07:14]
And I find that to be so cool.
[01:07:16]
But like you just said, like, oh, yeah, like you forgot about it
[01:07:20]
because you don't have to think about it anymore.
[01:07:22]
It's just fun. It's so cool.
[01:07:24]
Yeah, I was when I initially started this out, I was playing with the GitHub
[01:07:29]
Open API spec.
[01:07:30]
And I think at one point I was getting during some of the experimentation,
[01:07:35]
we were getting up around like 10 or 15,000 lines of code, I want to say.
[01:07:40]
Maybe more than it was probably more than that.
[01:07:43]
Yeah, that's actually not as bad as I would have imagined.
[01:07:46]
It's not it's definitely not the biggest Elm module by far.
[01:07:51]
But I don't think GitHub's API is the largest API either.
[01:07:55]
So I do wonder if there are APIs out there.
[01:08:00]
There probably are that would that would break Elm if I generated it right now.
[01:08:04]
I would imagine there are.
[01:08:05]
I should try and find one.
[01:08:07]
I wonder if there are any listeners who know of one that is big enough
[01:08:10]
that would that would break the Elm compiler because the generated code is just too large.
[01:08:15]
Yeah, if you generate you generate your SDK, you only use the commands.
[01:08:20]
You never use a task.
[01:08:22]
That's a third, at least, if not half of the Elm module that is just completely
[01:08:26]
ignored and never shipped.
[01:08:28]
Yeah, in your code.
[01:08:30]
Yeah.
[01:08:30]
And if you expose all the decoders and you don't use them, that's fine.
[01:08:35]
And therefore, that's why it's fine for you to expose all those things as well.
[01:08:40]
And I can't imagine like if you do this in TypeScript, like how how do you avoid
[01:08:47]
bundling all that code with with your production bundle?
[01:08:52]
Right.
[01:08:53]
Yeah.
[01:08:54]
So there's like module that code in a donation.
[01:08:57]
Like if you don't import a module, it won't be imported.
[01:09:00]
So you need to split things up by module, but it also uses objects, right?
[01:09:06]
So anything or classes.
[01:09:09]
So if you do anything with a product, then that for sure, all the things
[01:09:13]
that are related to products are going to get shipped.
[01:09:17]
Yeah.
[01:09:18]
Like if you're if you're using Spotify, for example, maybe you never hit
[01:09:21]
put or artists, you never hit the artist endpoint, but every song has an artist.
[01:09:27]
So you need the artist decoder for every song.
[01:09:29]
Yes.
[01:09:30]
So regardless, that's going to get shipped.
[01:09:32]
So you want the banana, you get the gorilla and the giant.
[01:09:36]
Yeah.
[01:09:37]
Yeah.
[01:09:39]
Yeah.
[01:09:39]
I do wonder how that works from language to language.
[01:09:43]
I should I should generate trying to generate the square square SDK for
[01:09:50]
Elm and see how big it ends up compared to the other languages.
[01:09:53]
Because I know they ship like six to eight languages, I think it is.
[01:09:58]
I'm guessing it's on NPM and you can find it there.
[01:10:00]
See how big the API is.
[01:10:03]
It is MPM.
[01:10:05]
They ship a square.
[01:10:07]
It's just square.
[01:10:08]
I think square square up square.
[01:10:13]
It is unpacked.
[01:10:16]
It is six just shy of seven megabytes.
[01:10:20]
That's for the node SDK.
[01:10:22]
I don't know how big they they also have the Java one on Maven.
[01:10:27]
There's a PHP one Rails C sharp.
[01:10:32]
Right.
[01:10:32]
Okay.
[01:10:32]
So this is really specifically for node.
[01:10:34]
It's not for front end JavaScript.
[01:10:38]
That would definitely be a problem for your bundle size.
[01:10:41]
That's that's a lot.
[01:10:42]
Yeah.
[01:10:44]
That does I wonder though, like does that affect I don't know enough about various
[01:10:48]
languages that have start like like Java has a startup time and does the size of
[01:10:54]
your dependencies drastically affect the startup time of your server then?
[01:10:59]
It's a problem for for cold starts for serverless functions.
[01:11:03]
That's for sure.
[01:11:04]
It there's a there's a hard limit and it slows it down.
[01:11:09]
So yeah, so that is an interesting thing.
[01:11:12]
I never considered that.
[01:11:15]
So we talked about cogeneration for open API specs.
[01:11:19]
We talked about cogeneration for GraphQL.
[01:11:22]
What else is missing in that realm?
[01:11:25]
Like do we have protobufs, cogeneration that is missing?
[01:11:29]
Do we have other applications that that are not even related to SDKs?
[01:11:35]
There are.
[01:11:35]
I think there is.
[01:11:37]
I want to say there's a protobuf some some Elmstaffer protobuf.
[01:11:41]
Yeah, there is actually.
[01:11:44]
What is it?
[01:11:45]
There is one for NATS that was just released this week.
[01:11:51]
I don't know what NATS is yet, but I am kind of curious.
[01:11:56]
I think there's gRPC is another one.
[01:11:58]
I think I've seen something for that realm.
[01:12:01]
There's always a few new ones here and there.
[01:12:03]
There's one that's message something.
[01:12:07]
Web message or something like along those lines.
[01:12:10]
I think most have at least a minimal implementation in Elm.
[01:12:14]
I do.
[01:12:14]
So the other code gen aspect that I would like to explore someday.
[01:12:20]
Maybe maybe I'm not sure if I'm going to go to it before or after forms is testing.
[01:12:26]
I figure if I can generate the endpoints, can I not generate Elm test code as well
[01:12:32]
for those endpoints so that like if you were right, say you were using Elm program test,
[01:12:38]
you could send out your requests and have it respond with fake data essentially that is generated.
[01:12:46]
Yes.
[01:12:47]
Yeah.
[01:12:48]
I've thought about this for Elm GraphQL as well.
[01:12:50]
Like there's this factory girl Ruby gem where you're using these testing factories to basically say
[01:13:01]
it knows what kind of data it's generating.
[01:13:03]
You can let it generate some random things or give specific data in some of those pieces.
[01:13:08]
But that would be very interesting for sure.
[01:13:11]
But generating a nice API for it is a challenge.
[01:13:15]
Yeah.
[01:13:15]
Also like generating results that make sense can be difficult, I think.
[01:13:20]
For instance, if you like create a user, then you need to return the same data that you gave to generate the user.
[01:13:31]
So the same name as the input, the same age as the input, but a new ID and that ID has to be different
[01:13:38]
from all the ones that you created previously probably, right?
[01:13:42]
So like because with Swagger back in the day, there was the ability to generate test code like this.
[01:13:53]
But in some cases, it's not exactly what you want, especially if you want things to be connected in some shape or form through IDs or something.
[01:14:02]
It gets tricky.
[01:14:03]
So very challenging, I think.
[01:14:06]
But if it works, super interesting.
[01:14:09]
Yeah, maybe I'll do that after form generation.
[01:14:18]
No, no, no, I'm still motivated to do it.
[01:14:20]
It's more what given a year, what can I do in a year and what will be the most beneficial?
[01:14:31]
Testing is still very beneficial, but I think more people benefit from forms than tests is my guess.
[01:14:39]
I think the testing side is more, I think there's an interesting challenge there and also provides a benefit, whereas forms,
[01:14:46]
it's mostly benefit.
[01:14:48]
Like the challenge I don't think is enormous or insurmountable, but the gains that people have from it are significantly higher.
[01:14:58]
Also, if you wanted to make it work for an program test, you would need to generate a command, a task, and some kind of effect
[01:15:08]
that goes well with the effects that the user defined.
[01:15:12]
Yeah, yeah.
[01:15:14]
I think you might just consider having like a
[01:15:18]
wrapper function where every endpoint calls that function to generate a whatever, whether it's a command, a task, a backend task, an elm concurrent task,
[01:15:30]
an effect, whatever it might be, like just
[01:15:34]
let the user define that's using the code gen part,
[01:15:38]
define a function that takes the decoder and headers and
[01:15:43]
HTTP method and all that as input, and then turns that into a something,
[01:15:48]
and then you let them do that. The hard part is in the generated code, then having the appropriate
[01:15:54]
type signature for all of that generated code, which you kind of want, and that becomes a pain.
[01:16:00]
It is, it's a type gymnastics, basically.
[01:16:04]
Absolutely.
[01:16:05]
You have four hours and you are not allowed to use more than three type variables.
[01:16:14]
Well, the exam starts now.
[01:16:16]
Well, Wolfgang, this is a really great asset for the community. So, thank you for pushing across the finish line. I really do believe
[01:16:25]
the more pieces like this we have in the ecosystem,
[01:16:29]
the
[01:16:31]
fewer caveats there are to like, oh, but does the Elm ecosystem, does it have good GraphQL support, does it have good
[01:16:37]
OpenAPI support? So, thank you.
[01:16:41]
And if someone wants to get started, what's a good place to start, where can they learn more?
[01:16:45]
They're completely,
[01:16:47]
if they want to go down the road of learning OpenAPI, I would say just Google OpenAPI.
[01:16:53]
Don't confuse it with OpenAI. You and I have been doing it lately.
[01:16:57]
Yeah, OpenAPI.
[01:16:59]
If you are looking to use it,
[01:17:02]
check out the NPM package.
[01:17:05]
Most likely, if you're looking to generate, and then whatever API endpoints you're, whatever service you're looking to communicate with,
[01:17:14]
generate something.
[01:17:15]
If you're looking to contribute to new features like form generation or testing, if you're feeling ambitious,
[01:17:23]
feel free to reach out to me on Slack or Discord,
[01:17:26]
or however else you know how to reach me. Amazing. And the real world
[01:17:32]
example that you have where you're hitting the API using
[01:17:36]
using OpenAPI is also very handy. So, worth taking a look if you want to see what it looks like calling these
[01:17:43]
generated functions. Thank you so much. Wolfgang, great having you on. Thank you for having me. It was great.
[01:17:48]
And you're in. Until next time. Until next time.
[01:17:52]
Bye.