elm-concurrent-task with Andrew MacMurray

elm-concurrent-task with Andrew MacMurray

Andrew MacMurray joins us to discuss `elm-concurrent-task` which allows you to run JavaScript functions with a Task style API.

PublishedNovember 6, 2023

Episode#94

Andrew MacMurray (github) (site)
elm-concurrent-task
lobanov/elm-taskport
elm-pages v3's BackendTask API
elm-review-simplify
0ui/elm-task-parallel package
elm/core Task API
lamdera-program-test
elm-pkg-js - A standard for shipping simple JS with Elm Packages
elm-concurrent-task pipeline worker example

Transcript

So right before recording you just admitted that you didn't have a pun.

I think that's going to disappoint a lot of listeners.

So what we can do is we can both concurrently try to come up with a good pun.

That's going to be our task.

That's the task you've set for us.

I see what you did there.

I see what you did there.

The task may have failed, but we might have a little bit of failure recovery,

some error handling on this.

Let's try to end then and move on from this sequentially.

I highly approve of these puns.

Andrew, please save us from these puns.

We've got Andrew McMurray joining us.

Thanks so much for coming on the show, Andrew.

Thanks for having me.

Pleasure to have you.

And today we are talking about your LMS.

Elm Concurrent Task Package.

So, well, to start us off, what is Elm Concurrent Task in a nutshell?

Like, what even is it and what does it do?

Well, it is, I guess you could call it, it's a task-like API.

So if you've used Elm Cause Task, it's designed to look very similar.

The main difference, I'd say, is that Elm Cause Task runs,

it's map to map.

Three and friends, it runs each of those one after the other.

Whereas the name implies an Elm Concurrent Task, it runs them at the same time.

So you can run like a tree of these concurrently.

The other thing it does is it also provides a convenient JavaScript FFI.

So you can call, the idea is that you could call a sequence of JavaScript

functions and chain the results together.

This has also been called task ports in the past.

If you've used Elm Pages v3 and you're familiar with backend task, it is basically a standalone

or more or less a standalone implementation of that.

And one that can be run on the front end too, which backend tasks as the name implies, cannot be.

But I will add, it was fully inspired by backend task.

You gave me the idea, Dillon, and I just took him around with it.

And I, on the point of running concurrently rather than sequentially, I just want to maybe give people an example to, to motivate that a little bit because, you know, it sounds, it sounds pretty in the weeds, but if you think about it, if you say, if you have two tasks defined for making HTTP requests, you know, to get the user users profile information and get the current weather.

There's some latency for that.

And, you know, maybe it takes some time to process the request on, on these different respective backends.

If you do test on map two, it's going to run the first one.

And then once that's done, it's going to go and run the second one, which is not great.

So it's a serious problem.

Why isn't that great in this case?

Well, because you, you stack the latency of the one on top of the other.

You, you turn it into a, you turn something that doesn't need to be a waterfall, i.e.

stacked one in front of the other finishing before the next one starts.

You, you could just fire them off both at the same time.

And that's sort of like the basic structure of, of web browsers is designed and, and of JavaScript of the JavaScript runtime.

It's designed to be really good at doing things asynchronously.

So it's at its best when you can sort of.

Fire things off and let them asynchronously come back and continue on.

It's not good when you hold up the event loop or, or waterfall things too much.

I actually recently learned that all the map functions are sequential.

Like I thought they were concurrent, but no.

It's surprising, right?

So in the example of the.

You make two requests, one to get the time or the weather and one another to get some other information.

Doesn't matter too much.

It's just slower, which has some impact obviously.

But if you like, you make a get request and another request to a post request or put a request to actively do something and you expect those two to, to be done, even if the first one fails, then that changes the behavior.

You could always, um, Hmm.

You know, I'm not even sure what task dot map two and in the Elm core task definitions would do, but yeah, maybe it would not fire the second one.

You could always do.

It doesn't, it doesn't like it, it actually does like a task that, and then on the first one, right.

And then it runs the second one.

So nothing gets sent out or triggered before you, before the first one completes.

We just had a new simplification in the arm review, simplify that if you have like task dot map three and the second element, uh, is clearly something that will fail.

Then the map three gets changed to a map two and we remove the third or the fourth argument.

Like if it's a task dot fail or whatever.

We, which is not going to happen very often, but it's now you will have like something that tells you like, Hey, this is simple.

Um, simplifiable or here's maybe something you didn't expect.

That is interesting.

Cause generally the Elm standard libraries have very intuitive semantics, but this is definitely one that's surprising.

And it is interesting because it is technically possible to define, and there are packages out there that define a way to use the Elm task package to to do parallel tasks.

Um, so that is technically possible, but but yeah, so, so.

So the FFI part of Elm concurrent task, I think is also really crucial.

So, and, and now the idea here you talk about in the read me is, is a hack free of way of doing FFI.

I think sometimes in the Elm community, FFI is sort of looked at as a dirty word, but I mean, we need to interact with things outside of the sandbox of Elm in some way.

So it's a question of how do we do that?

So, so what does that.

Look like an Elm concurrent task?

What does that look like?

So very similar to backend task, funnily enough, the idea is you would define a function name in Elm.

So like a string, like the, the string function name, you give it, uh, a decoder effectively, like what you expect to come back from the function, some way of interpreting the errors too, and then an encoded set of arguments.

On the other side, the JavaScript side, there's a, there's a JavaScript package that goes alongside the Elm package that just looks up the stringified name in effectively like an object of functions that you provide on the JavaScript side.

So it's when I saw it in backend test, cause I was like, that's such a simple, elegant idea.

And it actually has quite a nice, very nice kind of back and forth between the two packages.

And I think the main, the main safety.

That you provide is like library authors is telling Elm when things like the function hasn't been defined or like if it throws an exception or doesn't give you back what you expect to give back.

You're sort of baking all of that bits where if you did it in an application without some of the wiring, you could get, you know, unpleasant errors, things that go wrong.

They use your stuff when you're interacting with ports.

And if you contrast that with defining a.

Port in, in Elm and running that as a command or a subscription, if you, let's say define an outgoing port to set an item in local storage, I believe if you, if you throw an exception in a port in Elm, it's just throwing that exception in, in JavaScript.

So it's not, there are no guardrails that are going to prevent that unless you add them in yourself.

So what would be the impact here?

Would it cancel all the other?

Tasks that are remaining?

There's some, so it would, there's, there's actually two kinds of errors that you can get back.

You'd, you'd receive an error back through via concurrent task, and you can define certain tasks where you expect an exception to be thrown.

You can say like, catch the exception and then lift it into an error type.

Or you can, there's, there's actually kind of a lower level, which are called unexpected errors where like.

If you're, if you're not expecting this, this JavaScript function to throw an exception and then something happens, it's kind of like it will cancel everything.

And then you get a descriptive error message on the other side.

So yeah, that's, that's I've been, I've, I've tried it out in a couple of, a couple of applications.

That's pattern seems to have worked quite nicely.

So I'm a little bit confused about something.

What happens if in one of the ports through concurrent task or just plain.

Regular ports on JavaScript side, what happens if there's an exception that is being thrown synchronously?

Does that then gets does that then cancel all the other tasks that are related to, or in the same batch of commands being sent?

Are those canceled?

Do you know that by any chance?

So with the, if you've got a batch, like a batch that's gone out and one of them throws.

It's not clever enough to like stop that.

The, the ones that are in flight at that moment, they've already gone through the port, but any, any, any followup ones from that will be canceled.

So those that are like, and then, or map on that one will be canceled, but the other batch commands will just go through.

Which I think makes sense.

There's no reason that maybe they're disconnected.

They I'm doing.

One thing and another totally unrelated thing.

And if the first one fails, then I don't want that to be affected.

It depends on the case, but I guess having the possibility to say, well, these are connected and these are not connected.

Let's have them be separate just in case.

So that's, that's interesting.

If, if it's something that folks are actually interested in this definitely using like timeouts and HTTP, because one of the things that to do to make it compatible.

With well fill in the functionality of existing Elm core tasks, because that's, that's one thing that they're not actually compatible with Elm cores tasks, because they're from

many different types, but re-implementing HTTP timeouts, there's something called an abort signal in the fetch API.

So that may be, that may be a way of like, if people want cancelable, say like you've got a batch of like a hundred tests that have gone out.

One of them fails.

And you actually don't want any of the other ones to carry on.

You could, then maybe there's a way of sending them an abort signal to say, just stop.

Like, so you kind of reclaim memory or they may be they're expensive HTTP requests that you want to stop.

But I, I totally agree with you saying like, it's probably a case by case basis, like necessarily want that on every single task you define.

So you said something interesting there.

You had to re-implement HTTP task.

What is that all about?

So they, the concurrent task, like task type is not, uh, unfortunately it's like, it's not the same as the Elm core task type.

So if you wanted to use like time dot now or process dot sleep from Elm cores library, they wouldn't, they wouldn't fit in.

So the it's actually, I mean, I took the idea from Dillon as well, who did the same with backend task and that you, you write.

So at the concurrent task HTTP actually calls the fetch API underneath under the hood,

which is actually, which is actually nice because, you know, the built in Elm HTTP package uses,

uses XHR requests, which is kind of an outdated standard.

And there are some performance improvements and modernizations.

They're subtle, but there are some minor performance improvements

and general improvements in fetch.

So that's kind of a nice feature.

Okay, so that means that you had to implement some of the things in Elm

and some things in JavaScript.

And does that mean that your Elm package does not work without the JavaScript package?

Like, for instance, I thought that the JavaScript package was if you wanted to enable task ports,

but it just doesn't work without setting up that JavaScript library.

That's right, yeah, you do need the JavaScript library too,

because the JavaScript library will wire in the ports that you provide from Elm.

It's actually the thing that runs.

It runs the tasks themselves.

Right, so even if you wanted to run two HTTP requests

or two time.now functions that call out to the core one,

you wouldn't be able to run them concurrently

because you need a new primitive to run them concurrently.

Yeah, so it's basically using the exact same mechanism

for these internal primitives.

Primitives of, you know, concurrent task.now and HTTP

as a user would use to define their own custom task definitions in JavaScript.

Basically, the only difference is that when you call the code in JavaScript

to set up your additional definitions for concurrent tasks,

it already comes with some sort of pre-installed sort of core concurrent task definition.

That's the only difference, I think.

So how do you actually make those tasks concurrent in JavaScript?

Do you wrap everything in a set immediate callback

or I don't even remember the name of the function?

So it's basically you're sending out,

I call them internally, I call them task definitions.

So it's like that record of the function name,

the arguments that you're sending.

You send those all out in a command.batch.

So it's like everything goes out the port whenever it's ready to be executed.

And then the JavaScript runner will just run them.

And then once it's got the results, it calls an incoming port with an ID.

The main challenge of that package and the core bit of it

is actually just attaching IDs to all of the tasks

and then sending them back through and being able to reassociate them.

So because Elm, as it stands, can run lots of things concurrently,

like command.batch will let you run as many things as you can imagine

or that the program will take.

But the problem is actually being able to associate them with the previous call.

So that's all the wiring that this package is doing

to make it appear like they're just happening one after the other.

And as far as set timeout and things like that,

the way that JavaScript works,

things are asynchronous in the event loop by default.

So when you do fetch,

you don't have to do anything for that to be an asynchronous task.

You just fire it off and it's going to be running

and then continue the event loop.

But the set timeout thing is more of a hack

or the run immediate or whatever.

Those hacks are more for getting the event loop,

to tick, to try to hack the processing of things in the event loop.

So the example that I had in mind where things are not async by default

is the time.now command

because that just calls date.now in JavaScript

and sends it back to the port or to the message in Elm.

So that one is not asynchronous by default.

So do you wrap that one back into a set timeout?

Or is it fine if it's just synchronous?

It's not currently wrapped in a set timeout.

I think there's an inevitable very small amount of lag probably

in that it has to be sent.

It probably does this just by default of it having to go out of the port

and then back in.

There'll be a very small...

I don't know actually how I'd measure the lag that it's got on it.

I mean, it's also like, we're not...

I mean, it's not like we're interested in the moment of when the update function returns, right?

It's when you get a message back.

Like you will have some time between the two,

between the return of the update and the start of the update function again.

And thankfully, we're not that...

We don't need to be that precise in browsers usually, hopefully.

Not often at least.

I think there are like...

There's asynchronous for like things performing in parallel.

And then there are like the semantics of the sort of chain of concurrent tasks.

And if, you know, assuming that it's similar under the hood to Elm Pages,

it's basically like able to just resolve to check,

is this task complete and continue the chain of remaining tasks

based on which ones needed to be completed before it continues.

That part of it is not using this sort of JavaScript event loop.

It's just checking whenever it can, if it's ready for the next task.

Another comparison between commands and tasks,

I always find that that's one of these kind of almost rough edges in Elm

is like there are these two similar but different ways of doing things.

There are different ways of executing things.

And the semantics are a little bit different.

Like a, you know, task has an error type and they're chainable

and they're sequential, but they can technically be done in parallel.

And you might get a response, which might not...

Which is not the case for...

Well, you will always get a response, which is not the case for commands.

That's an interesting one.

I think that's like the main reason why you don't do that.

have what commands and tasks are different and then the other strange thing is like there are

only a handful of things in elm that you can use to create like a native task like http and some of

the you know dom operations give you tasks but then a lot of things there just is no task version

of you just do a command for example ports like you define a port and it gives you you know a task

for an outgoing port but i think a lot of people say well but i want to be able to

have a perform an http request and then take some decoded data from that http request and

write that to local storage and then i want to go and perform an http post request and then i want

to do this so like but you can't chain an http request with a port because

It's a command.

So I've always felt like sometimes when people are complaining about Elm being very limited

in the way you can do FFI, that there's no FFI in Elm.

You can only do ports or web components, right?

Those are like sort of the two standard ways to do communication with JavaScript and Elm.

Well, I feel like one of the biggest pain points of that, at least for me personally,

is just that it's very awkward to do that with a command.

I want a sort of chainable task style way of doing that.

I want to be able to include that in a chain with HTTP requests and other types of tasks.

So I think it's very cool that you've sort of built this abstraction that with minimal

hacks gives you a way to kind of do these different things all in that same chainable

With minimal JavaScripts.

Would you say it's a hack, Andrew?

The only reason it's not a hack is comparing it to the other ways that you have been done

in the past to make this kind of thing work.

Those are definitely hacks.

Why don't you explain a couple of those approaches that other tools have used?

There was one that I used in the past on an application that did inspire some of the work

on this package.

There's a package called Elm Taskport.

And it's a very clever hack.

And the idea is that you monkey patch XML HTTP requests.

So you add, like, you modify that global object to whenever you're sending off a HTTP request

with, say, you've got like a special URL scheme.

You might be like Elm custom function.

Your monkey packs version will.

Check for that.

And then you can call custom JavaScript from inside there.

And it works really well.

But you're modifying, like, global objects, which is a little bit dodgy.

So I think that's, like, that's without a doubt a hack.

Don't let your mother know about this.

But what your mother don't know won't hurt her.

I mean, if you whack your mother in the back of her.

She won't know.

Yes, it will hurt her.

What is that idiom?

Like, that makes no sense.

Don't modify global objects.

If you can help it.

Just don't hurt your mother, Dillon.

But that is the interesting thing about this approach is that.

I mean, your read me.

Andrew talks about this as FFI.

And conceptually it is.

But it's actually just doing this, like, accepted thing in Elm, which is like passing in some flags.

And, you know, it's not using any frowned upon techniques to do these things.

It's just an abstraction that makes it feel a little bit more natural to interrupt with JavaScript.

I think the main thing that really I found very interesting with that approach was you can use it.

You can use it in Node or in the browser.

Whereas if you've got if you're relying on the hacks like XMLHttpQuest, there's another hack with service workers that you can do.

Like they work great on the browser.

But if you wanted to run it in like, say, you wanted to run it in Dino or Node, like you have to add polyfills in and you're getting into a real fun situation at that point.

And now let's imagine that's Elm got a new release of ElmHttp, which now uses fetch.

Well, now you can't use that hack anymore.

So aren't you glad that you can that it uses XMLHttpQuest?

And then actually, if you want to use an Elm platform worker with Node.js, you also have to use a sort of polyfill to make XHR requests work in Node.js with Elm.

So this one in a way.

Otherwise, HTTP requests will fail.

I was like, I mean, if that's the case, and I would have known about this, but because Elm reviews is using Node.js, I'm not making any HTTP requests from the Elm app.

So there you go.

That's why I didn't know where I forgot about it.

So if we go back to that weirdness between commands and tasks.

So the way that I always understood it is with a tasker, you always have the you always have.

A response guaranteed or someone guaranteed.

So if you call time that now, then it will respond right away with the date.

If you do HTTP request, then it will respond when the response comes back or after a time out.

So one way or another, it will always come back unless the user exits the page before end.

But the problem with commands is especially.

With ports, you don't you don't have the guarantee.

So you can't make a request to a port and expect a response to come back.

I don't know if it practice it matters much because I don't know if it's a problem if there is a task hanging waiting for a response which will never come back.

As soon as some things are implemented by the user, you don't have the guarantee that it will come back.

But what I can imagine is that.

If you do like if we if we know that everyone uses concurrent task and that scheme and everything is set up as it should, then the line between what tasks and commands are useful for slims down a lot.

And I don't look don't know if there's actually any reason not to have the same API for both.

Well, one thing that I'm going to test doesn't help is sending a request.

And not expecting a response.

That is the only command is the only one that can do that.

Not not with many things, though.

But yeah, I think the what you could do in our current task seems like the pattern.

If you say you do something like log to the console, like you just return unit an empty tuple.

It is kind of like it's sort of a response.

But yeah, that's what you do in a lot of things with the Elms tasks API as well.

But I don't know if there's a real reason for all of them responding.

Maybe it's just like, well, if we have a way to send a task request and not get a response and some of the APIs get weird, maybe that's it's like, oh, well, we just did this entire HTTP chain function.

And then at the end, because you use this function now, we won't get a response.

And that's unexpected.

That's just like, oh, that's a bit weird.

Like it's a foot gun is what I mean.

Yeah, you're right.

It does give you the option to before like there's always the guarantee that tasks will complete.

Whereas you're not if you if you can write them yourself, you can break those guarantees.

Yeah. Although there's there's always sort of an illusion of guarantees any time you're working with JavaScript, right?

Like JavaScript is inherently something that you can't explain.

You can't expect to be well behaved.

And as anybody who has used it knows, I think.

And so, for example, it can throw exceptions like that's just a thing that JavaScript code can do.

And so you have to handle that for that.

And it may also time out.

That's like another class of poorly behaved behavior it could fall into.

But you could have a framework like.

So you could have a framework like Elm concurrent task, wrap the thing into a try catch.

And have a timeout.

And then you're sure to have a response.

Will the timeout be long enough?

That's a different question.

And like, do you also want a response that is a timeout?

Plenty of cases like I think you would prefer not have the response.

But you would prefer to have a timeout or at least it could be possible.

So one thing you mentioned as well is when you're getting back the data from JavaScript, you have a decoder.

How does that work?

And what happens if the decoding fails or do you need to do?

Do you just get a JSON encode value and you need to decode it manually?

So you get back a JSON encode value.

Right under the hood, it will be a JSON encode value.

The JSON decoder gets run on it.

If it succeeds, you get the value back through your.

I've called it in the readme like the task flow, which is like your kind of regular error success flow that you do when you're chaining tasks together.

If there's some error handlers, there's some functions that you can use to say, like, if you get a decode error.

Do something with it.

You can either, like, fail with a custom error or you could, like, recover it and lift it into your, like, success type.

But if you don't add that in, there's what's called, like, an unexpected error gets returned out through the on response callback, like a message that you provide.

And that stops everything.

You mean it gets sent back to JavaScript again?

So this is all Elm side.

Like, it's not.

On the Elm side.

On the JavaScript side, it's all, like, it doesn't really know anything about it.

And the ports are just, like, just send me, like, JSON encode values back through.

So it's very those ports are very safe from, like, I think you'd have a very hard time to make them blow up.

But then on the Elm side, it's all handling the, like, what do you expect to come back from the task?

And if it does something weird, it's, like, stop.

Like, abort the task chain.

Like, don't send any more values back out through JavaScript at that point.

So if somebody wants to wire this into an app, what does that involve kind of getting it set up with the boilerplate?

What do you mean?

Basically, what do you need in your model?

What do you need on the JavaScript side to wire it in?

What other messages and all that sort of thing do you need?

So you need a couple of things.

There's in your model, there's what's effectively the task, the tasks model.

I've called it a task pool.

The name could change.

But it's an idea that you can have multiple.

Yeah, you could have, like, multiple tasks running at the same time.

So it's, like, this is something that's managing all of the state internally of the in-flight tasks.

And you might want to swim in it, a task pool.

So you need the task pool.

There's two messages.

So there's one that I've called, like, on progress.

So that's kind of, like, the internal wiring that's, like, every time you get, like, a response through the port, it's, like, kind of funneling that back through.

It's just, like, updating the progress of the task and then giving the next command to be performed.

And then there's one message which is, like, when everything's done, like, you've got, like, a final result from the task, which is either, like, an unexpected error or the success.

And then there's just some subscriptions to wire all of that up together.

So it is a little bit of boilerplate.

The advantage of the core task or command is all that, like, wiring is hidden away from you.

So you can just fire the task and then give it, like, a callback, like a success callback.

Then in, like, a command's case, it's just, like, just fire it and then stuff will happen.

So, like, you abstract away a few of the details of the JavaScript wiring.

So, like, you initialize your Elm application and then you can, you pass in a JavaScript object with all of your port definitions.

And your port definitions are, you know, instead of, like, going through that dance of the incoming and outgoing ports and wiring those in and adding subscriptions.

And then if you have a subscriptions port that you stop using and then the Elm compiler or the JavaScript code crashes because it's no longer defining this port because it's unused code in your Elm code.

And all these, like, rough edges kind of go away.

And instead you just are defining these ports, these tasks for Elm concurrent task in a set of key value pairs.

You give it a name.

And you give it an async function or a synchronous function.

And then you can just go back and do it.

So, you can do a synchronous function if you want in JavaScript.

And return some data and give it a decoder.

So, like, that feels like the right mental model for my brain to interact with that.

So, it's a little bit of wiring, but it cleans up a lot of things as well.

That's good to hear.

It's good to hear the mental model fits.

I've tried to abstract as many of those wiring details away.

But there is inevitably a bit of.

Spoiler plate there.

You just can't escape.

In this community, we have long accepted boilerplate as being okay.

Like, sure, if we could use less, we would not say no.

But, I mean, we're okay with it, right?

I think for me, it's like as long as I don't have to think too much about the boilerplate, then I'm happy with it.

Like, I don't want to be figuring out complicated logic.

It's just like.

Give me a list of things I need to provide and I'll do it.

Actually, like, there's one thing that I'm wondering about.

So, in the JavaScript, you set a, you call a function called concurrent task dot register.

And you give it your ports, your input ports and your output port.

And a list of tasks.

So, those are the, that is the object which contains names of the functions and then what to do with the arguments.

That were sent.

And every time you need to ask to add a new task, you need to both do it in the JavaScript side and do it in the Elm side.

And you have to make sure that those are always in sync, right?

Have you had any problems so far with them?

Like, oh, I forgot to set up one up or I had a typo in one of them.

I'm guessing not, but.

I mean, that inevitably happened.

I spent a bit of time wiring it into, like, an existing, like, running application that I've got.

And you spell the names wrong.

You're like, oh, damn, I missed that one.

It gives you, so it will give you back an unexpected error at that point, which is, like, it says, like, I couldn't find, I couldn't find a registered task with this name.

So, it's not perfect.

You can, like, you can mismatch the names.

But it does give you some hints to it.

You can say, like, you might have a typo or maybe you forgot to add it into the JavaScript object of, like, your list of tasks.

So, this is, like, one of the things that I would like to do with Elm review.

And I put it on pause, but it's something that I'm pretty close to having done, which is having Elm review look at both Elm code and other files.

So, you could have, in theory, or it would work with this new feature that I'm working on.

That I have somewhere stashed up in a Git branch.

You could have a rule that says, I want to look at all the Elm code, but I also want to look at this specific JavaScript file or just all JavaScript files.

And it could look at the list of tasks that you define in your call to concurrent register.

And then it can compare that with what you have in Elm on the Elm side.

And it could figure out, like, oh, I have this.

It could figure out, like, oh, well, the one you call on the Elm side does not exist in JavaScript.

Or even, like, oh, you have a task on the JavaScript side that is defined, but that is never used in Elm land.

So, that's something that I would like to see happen at some point.

Just, like, an extra layer of guarantees.

Like, you're sure that you have boilerplate, but it's done correctly for sure.

I would like to see that.

Because that case where you, that's an interesting one that you mentioned where you can't know whether it's unused.

Even if they do match up, you're not sure if there's no way of figuring out unless you've got something like Elm review to tell you, like, you can get rid of these two.

Like, they're never called.

The less JavaScript, the better.

As long as you put as many rules as possible, then it's fine.

Just exchange, like, 10 lines of JavaScript for 10 lines of JavaScript.

And then you can just write it for 10 Elm review rules and, yeah, it will be worth it, right?

I'm obviously biased.

Something doesn't sound right here.

So, this might be less of a 1.0 thing, Andrew, and more of a future thing.

But I'm curious, especially as a maintainer of a similar mechanism who's been thinking about these things as well.

What are your thoughts on unit testing something like this?

I think there are ways to make it doable in a pretty nice way, but it takes a little bit of introducing some abstractions to make that possible.

Is that something you've thought about?

Very interesting you mentioned that.

So, I had the same thought.

I was like, I really wish I had a way to try these in isolation and, like, what are they doing in the real world?

And I wrote for that runs on CI.

And I wrote for that runs on CI.

And I wrote for that runs on CI.

And then they changed what I'm doing.

But I've got, like, a little integration test runner that's, like, a custom.

It's kind of a custom program for it.

And it's not fancy, but it, like, it does the trick.

Like, it's kind of a simple way of saying, like, I expect this value to be a success and you can match on this.

You can assert on, like, how long the tasks take.

So, I've got one that it was actually very hard to test the batch.

It was very hard.

It was very hard.

It was very hard.

So, this batch implementation of, like, running a very large, very large list of tasks, running that in Elm test for some reason, which I haven't figured out, it does not enjoy it.

It really, really struggles.

But if you're actually running it in a real application, it goes very quickly and doesn't have any problems.

If anybody knows why, that would be, I'm all open to suggestions.

But having that confidence, being able to actually run the tasks in, like, a test suite was really, really, really hard.

I mean, I think that the fact that I had to run all of my tasks in, like, a test suite was really, really valuable to make sure that I didn't mess any of the wiring up or stuff was coming back strange.

So, that is a JavaScript test suite, right, which runs the Elm or partial Elm application maybe?

So, it's actually just an Elm application.

And I've written, it's like a single, like, a platform worker application.

And you define, like, a list of tasks in your, like, in your test file.

And then you're just passing those to the runner and it runs them.

And then you can, there's some Elm functions to match on the results that come back.

It's not, it's not packaged up at the moment.

But if it's something that folks are interested in, like, if there's a need for it, I could definitely think about extracting it out into something that's a bit more usable in other projects.

My initial thought would be to use Elm program test for this.

And then have a library that makes Elm concur task work for Elm program test.

And where you would mock the JavaScript responses or the HTTP responses and so on.

And I think that would work quite well.

And it would probably work in Elm test.

Maybe you should open an issue for that problem you've had if you haven't already.

But, yeah, you're right that you're mocking the JavaScript side.

So it wouldn't be as reliable as your integration tests.

I mean, integration tests are usually more reliable, just not as precise as unit tests.

That's a very interesting idea.

It makes me think of, like, Elm program test, obviously.

And then Martin Stewart's Elm program test for Landerer where you get responses from the backend, interactions with the backend, with the frontend, and all those kinds of things that you could simulate.

And that's, since it's all just Elm code, it's fine.

I haven't seen that before, the Elm program test for Landerer.

It's worth checking out.

There's a great talk where Martin walks through using the tool and shows the visual runner with all the connected clients.

It's very cool.

And, yeah, it gets really interesting, too, if you had some way.

I really like the Elm program test idea as well.

And, like, if you had some way to even have a few sort of clients.

I mean, I think it's a really good idea.

And I think it's a really good idea to have a few sort of core implementations for things like local storage where you could, you know, you could imagine if you wanted to get really fancy with this.

You could say, given this initial value in local storage and then let it actually simulate for the setting and getting in local storage.

Let it actually simulate that.

And now you could have Elm program test.

You could even have a way to expand some of these definitions.

Where there's, like, a certain set of core web platform primitives that you're able to simulate in a fairly realistic way.

So, it's really interesting, man.

I think you could go one or two or ten steps further.

Like, if you try to simulate HTTP, you just simulate a browser and DNS systems and everything.

And, I mean, how hard can it just be to simulate Google.com?

Any Elm package?

You can do it, Andrew.

I will say it too after this call.

Assuming that there's no free will and that the universe is deterministic and not probabilistic, in theory you could simulate the entire world and all its behaviors in a pure way.

Just as a pure function.

So, that's if you want to go a few steps further than that.

I think that's a good idea.

I think that's a good idea.

I think that's a good idea.

I think that's a good idea.

I think that's a good idea.

I think that's a good idea.

I think that's a good idea.

I feel like you went from giving him, like, oh, this is not for V1 but this could be for V2.

And now we're, like, can you just reimplement all of computer science please?

So, one thing we haven't asked.

you about or maybe not enough like what kind of use cases did you actually use this uh

package for like when did you feel the need for something like this yeah that's a good question

i so my my initial need for it is about a year and a bit ago i was working on a small medical app

and i wrote the back end in elm as an experiment and thought this is great it's lovely just just

such a nice pleasant experience and i'd set it up where each hdp endpoint was effectively a task

and i was using elm task port then and everything it was working great i was really really happy with

the with the the readability and like how easy it was to change stuff but the only problem i found

was that these tasks were performed one after the other so all of the subtasks and there were a

couple of things that i had to do to get it up and there were a couple of i had it in a cli application

as well that was kind of doing some background stuff on like there was some sort of like work

processes that was running and it just made me think this would be so much nicer if i could run

a lot if i could batch up a lot of these tasks at the same time so and then i did that and now

that's all really fast and works exactly the same so well almost exactly the same it's yeah so that's

i from my perspective i'd say like these kind of bucket like if you're using kind of a back end

as like hdp endpoints this like clear sequence of tasks i've found and like a cli application if you

got a cli command like very much like elm pages scripts it's the same kind of idea like those

i found those use cases really they fit really nicely there yeah because you want your server

to be as stateless as possible

ideally or each endpoint but then because you have to chain tasks that go through

transform to command now you need to store the state of ongoing requests in the model and yeah

that doesn't feel very good you can imagine i definitely tried that to start with and quickly

got oh my goodness this is it's just not a nice it's definitely not a nice way of writing for

exactly as you say like a stateless a stateless process

so it's it's basically the the pattern of a functional core imperative shell which i think

is a really nice model and it's like so elm is a really good sort of central processing unit

brain for like the the hub of everything and then the messy details as needed you can

delegate out to javascript code and it can do whatever you need it to do and you can define

an asynchronous code that's you know just going and doing what it does best like firing something

off and coming back on the event loop when it's done and not holding up the event loop from

continuing while it processes those things so yeah actually like elm is pretty nice for for

using for these sorts of uh functional core imperative shell ways of scripting doing some

sort of back-end work or or running in your front end

So why did you call it Elm concurrent task and not Elm script?

Like, just like JavaScript.

Elm script, yeah.

I actually wouldn't be surprised if there was already a project called that.

Yeah, probably, actually.

Sorry if I don't remember you, author of Elm script.

I am curious about the word concurrent.

So it's the same thing as parallel.

The exact same thing.

Let's open up that can of worms, Andrew, if you'd be so kind.

To be honest with you, my initial calling of concurrent task is there's already Elm

task parallel as a package out there.

One that would be very confusing.

Like, which one do I use?

I think the other.

The reason I add on the side of calling it concurrent was that technically

JavaScript is single threaded.

So it's a, it's a, it's a, it's a technicality that, um, yeah, is that

technically things in that environment cannot run in parallel.

Like true parallelism is like on different cores.

Let's say, but they're not just parallel, but yeah, I thought the

That educational means doing the same task on different data at the same

time and concurrent just like two things are running at the same time,

but they might be doing it different things.

I'm right, but I also feel like people are

just gonna shut up.

Sorry, listener.

is, there's a thread.

There's a long thread on own Slack.

If it's still there, I

I feel like it made the same point.

And then people corrected.

There's a thread, just one thread,

or are there multiple threads on it?

I would have to reread it and process all that.

It's much of a muchness, I would say.

You could call it either.

You could call it parallel, concurrent.

To be honest with you, I think my main motivator is

I didn't want it to clash with the other package

that's already out there.

Just because it's like if you're reaching for that same name,

it's a small thing, but it's likely to cause a bit of confusion.

But I think your name is technically correct anyway.

So technically correct is the best kind of correct.

Yeah, so I've definitely thought about,

Andrew and I have discussed whether

backend task would be a nice fit for sort of using

the concurrent task API under the hood

and maybe even making it interoperable.

But it's definitely a similar paradigm.

And it makes me think also like as a framework,

Elm pages would have the ability to give you

a little bit of wiring,

like all of the boilerplate for adding the right thing to your model

And then also, you know,

you could be adding an incoming and outgoing port and.

And making sure that there's no typo in the names of.

All these little details and right.

Adding the right message.

They're just like a handful of little things.

But Elm pages could build in something like this for you.

Because in my opinion, like this is kind of like the ideal paradigm

for using these things.

So that could be an interesting thing to explore as well.

It's just I know it's a lot of work actually.

I think it's really important to be able to actually rewiring all of it in

because it's yeah, you've like Elm pages is an expansive API.

It's very impressive what it's doing.

But I yeah, it would be a lot of work actually wiring it in.

So it's only like 50 modules.

And it only took him like a month to release or something.

Something like that.

I think it's a bunch of credits.

It all goes to his head.

I wonder if there is a way we could do it incrementally like to get some useful feedback

on like is it providing the functionality that you need or are there some like weird

We don't have to like reimplement everything under the hood with that.

I'm not sure how to do it.

That's the tricky thing when it's like such a backbone of it and there are all these like

things that are.

Deeply integrated but but it would be so nice if like the community could sort of coalesce

on like one nice way to do this and yeah, I definitely think you've hit up hit upon

a really elegant formulation of this sort of thing that as a community we've been iterating

on for a while in some way shape or form and starting to feel like it's really coming together

as a as a paradigm.

Have you by any chance heard about the Elm package JS?

Approach from Lambda.

Do you remember much about that?

Because I have yeah, I don't but I feel like there's some kind of overlap here where basically

Lambda is a full Elm JavaScript.

Sorry full Elm framework.

So back ends and front end isn't in Elm and there are you can't use ports or at least

at some point you can use ports.

So there's no way to do that.

But I mean, I think there is a whole range of possibilities.

point you couldn't use sports but in some cases like it's necessary even just to use more recent

browser functionalities so mario mario rojic suggested an approach called elm package yes pkg

to make that work and i don't remember how it works i wonder whether it works somewhat similarly

like there are names that you can call and they are available as elm values or ports or

i don't remember maybe something to look at yeah and there was also a similar like

a goal i think of giving a standard way of you know if if somebody implements uh

something for local storage or copying to clipboard or these basic web platform adapters

you can sort of plug and play with different adapters that the community is maintaining

yeah and i mean actually like so elm packages would be a solution for that

i guess but i would have to look at it again but you could also potentially do that

with elm taskport like you already have a javascript implementation that just adds

arbitrary primitives and you make them available in elmland you you have the power to

add new primitives like that that is what this framework enables right yeah i've been working

on my own set of uh primitives that are useful for something like node i had it was interesting

i had um there's an example in the examples repo where oh sorry in the example in the examples in

the elm concurrent task repo there's like it's a little kind of pipeline worker so it's like it's

like a platform platform worker that will like list all the different tools in the elm taskport

listen it's listening out for like sqs messages and then orchestrates uh sorry like a amazon sqs

like a queue so it's like this yeah it's this thing like on a queue and then does like a bunch

of a bunch of things with various aws services and i had like i was quite surprised at how pleasant

it was to write so there was some very minimal bindings to the aws sdk so i mean that's a much

bigger example you don't want to go like full hog on that but actually having a set of like simple

primitives like you could have for the web platform or you could have with things on node like this

seems to work quite nicely and then you just use as dylan you were saying before you just use elm

as the brain effectively for orchestrating all of these together so it's interesting having a

standard i like the idea of having a some kind of standard way of distributing those two two things

alongside like the javascript

and the elm module yeah i mean like to me it's it's one of the things that's so at the heart of

elm and its design is this this separation of the clean pure elm sandbox from the rest of the world

but then to me the the current design of ports as a command subscription sort of thing

i know there is this concept of like the actor

model and and kind of sending something out into the world and not necessarily tying that together

to something that comes back in from the outside world and that that is an idea that i think evan

did design intentionally to like have it be maybe more fault tolerant like you know like elixir or

um erlang's sort of actor model for for how the erlang vm works

but i to me the thing that makes that's essential to elm is the purity of the elm sandbox

and all bets are off for anything outside of that in the outside world and this preserves that but

it makes it much more manageable to work with that and then similarly like if you can install

elm packages and they violate those guarantees and expectations that would go against the

that core design to me

um so if you could install a package and it has kernel code or it has ports defined but at the same time like we've been trying i think in the elm community for a long time to find a nice way to share a definition for

how you define something for local storage and these basic things you know how you use the

internationalization apis or apis that need to be built into the web platform they're important we need to use them for a lot of different things but i think it's a good idea to

as web developers um but how do you share them and the type definitions for them and stuff like that and we don't want

ffi bindings that we can just trust but i feel like there is something like elm elm pkg js elm package js

mario's specification is trying to address this problem in some way where it's like a shareable

definition that has an elm type associated with it and elm concurrent task does seem like it would

pair really well with the elm package js but i think it's a good idea to use this as a way to

nicely with that, like letting you define these tasks. And if you could like install a task,

that could certainly be very interesting. And again, other people may have different opinions

on this, but to me, that preserves what's essential about the guarantees of Elm, which is

you can't trust the outside world. You're still not trusting it, but you're making it a little

more convenient to use it in a way where you're skeptical of it being safe.

The only problem is then everyone would have to install Elm concurrent task.

It would be a quite critical dependency.

But it is interesting. Like, yeah, you don't really want to be re-implementing a lot of the

very low level stuff over and over again.

Right. Exactly. Well, Andrew, if somebody wants to get started playing around with Elm concurrent

task and learning more about it,

what's a good place to do that?

Because if you search on the Elm package website, Elm concurrent task,

that should point you in the right direction. The readme has some instructions on

installing the Elm package and the NPM package. And yeah, if you've got, please get in touch

if anything is unclear, like I'm available on Slack or GitHub.

Wonderful. Yeah. And I definitely recommend there are some really nice examples there as well that

are worth checking out. So yeah.

Wonderful. Well, thanks again for coming on the show, Andrew. It was a pleasure having you.

Thanks for having me.

And Jeroen. Until next time.

Until next time.