spotifyovercastrssapple-podcasts

elm-program-test

Aaron VonderHaar joins us to share the fundamentals and best practices for his high-level Elm testing package, elm-program-test.
February 15, 2021
#24

Transcript

[00:00:00]
Hello, Yaron.
[00:00:01]
Hello, Dillon.
[00:00:02]
And again, today we're joined by a special guest, Aaron Vonderhaar.
[00:00:08]
Welcome Aaron.
[00:00:09]
Hey, how's it going?
[00:00:10]
It's going well.
[00:00:11]
Thank you for joining us.
[00:00:12]
You want to say a quick hello and tell us a little bit about yourself?
[00:00:16]
Sure.
[00:00:17]
I am Aaron Vonderhaar.
[00:00:19]
I have been, I guess, a full stack developer my whole professional career and did a lot
[00:00:27]
of mobile iOS and Android development in the past and have been doing Elm, I guess it's
[00:00:32]
almost eight years ago now that I started writing Elm format and was working at KnowRedDNK
[00:00:39]
for about four and a half years doing Elm there and now doing some Elm consulting work.
[00:00:44]
Okay.
[00:00:45]
You're hired.
[00:00:46]
Perfect.
[00:00:47]
Great.
[00:00:48]
Yeah.
[00:00:49]
And we had a lot of fun discussing Elm format in our last episode and it's fun to have you
[00:00:55]
on to discuss another tool of yours.
[00:00:59]
So today we're going to be doing a deep dive on Elm program test.
[00:01:03]
So let's start with the definition.
[00:01:06]
What is Elm program test?
[00:01:08]
Yeah.
[00:01:09]
So Elm program test came about as filling a gap for me in doing Elm development and
[00:01:15]
doing it in a test driven development style way of being able to write high level tests
[00:01:21]
that are written basically from the perspective of a user or an outside entity interacting
[00:01:27]
with your Elm program.
[00:01:29]
And it basically gives a nice high level API to write Elm tests in that fashion.
[00:01:34]
In practice, I found the main benefit that that gives you is the ability to have test
[00:01:40]
coverage that is resilient to refactorings and specifically like significant refactorings
[00:01:46]
if you need to restructure how your application works.
[00:01:49]
If you're converting like a single page app into a spa app, bigger things like that, or
[00:01:54]
even just like sophisticated new features or changing workflows for your users, the
[00:01:59]
tests that you write with Elm program tests tend to give you test coverage that allows
[00:02:04]
you to do safe refactorings that are much larger than refactorings you'd have coverage
[00:02:08]
for using normal unit tests.
[00:02:11]
You mentioned that you were writing Elm tests.
[00:02:14]
So this works with the regular Elm test binary, right?
[00:02:18]
Yep, exactly.
[00:02:20]
This is basically just a sophisticated set of helper functions that can be used with
[00:02:24]
normal Elm tests.
[00:02:26]
Right.
[00:02:27]
So just to give like a high level overview of like, what the actual wiring looks like,
[00:02:34]
you pass in an init update and view function to Elm program tests.
[00:02:39]
So that's how it works is it's actually simulating, initializing and going through the updates
[00:02:45]
and responding to events in the view.
[00:02:48]
Yep, exactly.
[00:02:49]
Essentially I'm program test was extracted from some helper functions that I started
[00:02:55]
like adding throughout no red inks, a test code.
[00:02:59]
And the very first version of that basically just looked like we were creating a model,
[00:03:04]
piping it through a bunch of update calls to the update function with different messages.
[00:03:09]
And eventually that got fancier where I realized, Oh, we can use the HTML testing features of
[00:03:15]
Elm test so that we can actually inspect the view that is based on the current state of
[00:03:21]
the model and make sure that the view has ways to interact that we expect to produce
[00:03:26]
the message that we want to feed through the update loop.
[00:03:29]
So essentially that's the complexity in Elm program test is creating an API that essentially
[00:03:34]
is speaks in a different language than view model update, which is like the technical
[00:03:40]
implementation of how Elm programs are written.
[00:03:43]
And Elm program test lets you use functions like, Oh, click the button with this label
[00:03:48]
or pretend that this HTTP request you made came back with this error code, things like
[00:03:53]
that.
[00:03:54]
Right.
[00:03:55]
So let's talk a little bit about some use cases and when somebody would use this compared
[00:04:00]
to a regular test.
[00:04:02]
So we covered test driven development and Elm test in general in our Elm test episode,
[00:04:07]
but this is something different.
[00:04:09]
This is a higher level form of testing.
[00:04:12]
And so when would you reach for this rather than a more vanilla unit test?
[00:04:17]
Sure.
[00:04:18]
So something you two talked about in your previous episode was about the testing pyramid
[00:04:24]
and kind of the different layers of that.
[00:04:26]
I guess there's a couple of things that the main thing is that if you're happy with unit
[00:04:31]
tests you have Elm program test isn't trying to stop you from writing unit tests that you're
[00:04:36]
already happy with.
[00:04:37]
Elm program test provides the level above that and in particular, if you look at end
[00:04:44]
to end tests using something like Cypress or Selenium web driver based testing, my hope
[00:04:51]
has been that Elm program test will allow you to write tests in the same language as
[00:04:56]
those where you're talking about what the user does, what they see on the screen.
[00:05:01]
But if you take the assumption that you trust the Elm compiler and the Elm packages to actually
[00:05:07]
do what their documentation says they do when they're compiled and running in a browser,
[00:05:12]
if you can trust Elm as a framework to do those things, then you can use on program
[00:05:17]
test to write tests that cover those same high level features, but that run much faster
[00:05:23]
are completely not flaky, completely deterministic.
[00:05:26]
Yeah, no, if you can exercise some functionality at a low unit level, then that's ideal.
[00:05:33]
But that doesn't always give you confidence.
[00:05:36]
So if you, you know, one of the rules of thumb that I try to follow is the more high level
[00:05:42]
my testing becomes, the less comprehensive it should be.
[00:05:47]
So if I'm noticing that I'm trying to exercise every combination possible in every edge case,
[00:05:55]
and it's in a high level test, then I'm thinking there's something that either needs to be
[00:06:00]
split out into a module that's testable or simply exists that way, but isn't being properly
[00:06:06]
unit tested to give me confidence because that's really not, it's going to be extremely
[00:06:11]
verbose to have to create a whole test scenario where you're describing clicking through things
[00:06:18]
and all these permutations.
[00:06:20]
Because you know, just mathematically, if you think about a higher level test, you're
[00:06:26]
putting more pieces together.
[00:06:28]
So there are more variables, there are more things operating together on that page.
[00:06:32]
And so if you try to exhaustively check them, now you've got a combinatoric explosion.
[00:06:39]
And it's just not going to be effective, because you can't do that.
[00:06:43]
But you can at the unit level and you want to.
[00:06:45]
So that's one thing I try to pay attention to.
[00:06:49]
But then, like, so why not just test everything in isolated unit tests, right?
[00:06:55]
And the reason for that is that you don't get the same confidence about things interacting
[00:07:01]
together and about the system functioning at a high level.
[00:07:05]
Like what is the user's experience when they come in and they click on byproduct?
[00:07:10]
Do you get money?
[00:07:12]
That's an important thing to know.
[00:07:13]
And a unit test might tell you that the credit card information is being validated correctly.
[00:07:19]
It strips out white space appropriately, formats it, gives the right client side validation.
[00:07:25]
That would be a great unit test.
[00:07:27]
But is it making the right API request and handling the response, showing error messages?
[00:07:32]
You want to know that when a user goes to do some important business flow, that you
[00:07:38]
want confidence that it's actually going to do that.
[00:07:41]
And unit tests don't necessarily give you that.
[00:07:43]
Yeah, that's exactly right.
[00:07:45]
And in Elm, being a statically typed and strongly typed language as well, the compiler gives
[00:07:51]
you a lot of protection for some types of refactorings.
[00:07:55]
But Elm program test is really at the level that you can't get type safety because it's
[00:08:01]
looking at like, oh, when you have this sequence of events happening in this order, something
[00:08:07]
goes wrong.
[00:08:08]
And especially like in Haskell, you can get a bit fancier with types.
[00:08:13]
But in Elm, you like definitely can't write a type for your update function that says
[00:08:18]
that, oh, once this message happens, you can never get these initialization messages anymore.
[00:08:24]
You only get this later class of messages.
[00:08:26]
So Elm program test allows you to kind of in a more natural language, I would say, than
[00:08:33]
writing unit tests allows you to get coverage of those things.
[00:08:37]
I don't know if you've ever ended up with tests where you're trying to like initialize
[00:08:42]
a model, send it through a bunch of update calls, and then check the state of the model
[00:08:46]
at the end.
[00:08:47]
You can definitely write those as unit tests.
[00:08:49]
But I find that it's often very verbose.
[00:08:52]
And you have to end up putting a whole bunch of like, fake data and things that are really
[00:08:57]
implementation details.
[00:08:59]
Whereas when you write a test of that nature, what you want to be thinking about is, okay,
[00:09:03]
what is the flow of events?
[00:09:05]
And you don't want to care about how to construct the record of the data that represents the
[00:09:10]
JSON that comes from this endpoint and all those lower level details.
[00:09:15]
So you say that those tests are supposed to be easy to read.
[00:09:18]
Have you looked at Cucumber?
[00:09:20]
Or are you inspired by Cucumber and those kinds of testing frameworks?
[00:09:25]
Like I think it's BDD?
[00:09:26]
Yeah, I've been really interested in those techniques.
[00:09:31]
There's a talk called something like test driven development the way it was meant to
[00:09:36]
be done by Matt Wynn from and is probably like 15 years old now.
[00:09:40]
So L program test, I think is a layer that's needed if you wanted to write tests in that
[00:09:47]
style.
[00:09:48]
Because that BDD approach is really that you develop a domain specific like application
[00:09:54]
specific language to write these high level tests in, whereas talking about things that
[00:10:00]
are relevant to your business domain, like in no red ink, it would be like talking about
[00:10:04]
teachers and classes and assignments and creating new assignments.
[00:10:07]
Yeah, I think that's that's kind of just like a higher level DSL on top of like to implement
[00:10:13]
some of those features like to implement or in a BDD style test, you might say when the
[00:10:19]
user logs in, and then dot dot dot other stuff to implement that how does the user login
[00:10:26]
step of your BDD testing, you would, I think, ideally use something like program test to
[00:10:33]
say, Okay, click the login button, then fill in the name of the current user fill in the
[00:10:39]
password of the current user, click this other thing, then simulate the back end responding
[00:10:44]
to the post request with an okay status.
[00:10:47]
So that's kind of implementation specific.
[00:10:50]
But the way I just described those steps is still high level, it doesn't depend on the
[00:10:55]
details of how the form validation is implemented, or how they like what UI framework they use
[00:11:01]
to create the form.
[00:11:03]
So on program test is kind of high level for the technical side, but it's still lower level
[00:11:07]
than something like cucumber tests,
[00:11:09]
I have an opinion about cucumber, which is that now I may be totally wrong, or, you know,
[00:11:15]
reasonable people may disagree, but I'll share my opinion for what it's worth.
[00:11:19]
So what you were just discussing there, Aaron, with it being elm program test being a high
[00:11:24]
level way to express a flow that's not necessarily as coupled to the actual technical implementation,
[00:11:32]
not being as likely to change if you change implementation details, right?
[00:11:35]
In my mind, that is exactly what I want.
[00:11:38]
And the cucumber part of writing it in in actual plain language, I don't like and that's
[00:11:44]
what cucumbers for anyone who's not familiar with the Gherkin cucumber syntax, the sort
[00:11:49]
of idea of, you know, one of the concepts of behavior driven development BDD is that
[00:11:54]
you actually have customers or customer, you know, product managers, that type of thing,
[00:12:00]
in their writing cases, and it's not code, it's just text.
[00:12:05]
And you use regular expression, or various forms of parsing to do like language parsing
[00:12:12]
and say, when I log in as email address, and then you use some regex to capture that email
[00:12:19]
address and then log in.
[00:12:20]
And in my in my experience, that just creates a layer of indirection that doesn't make it
[00:12:28]
any more high level, it actually just makes it more confusing what's going on.
[00:12:32]
If you're, if you are not a programmer, and you're writing that, to me, it doesn't seem
[00:12:37]
like it's making it any easier just because it's an English language, because it's still
[00:12:41]
a specific syntax that you have to write, it's just one that's been built with this
[00:12:46]
layer of indirection of regular expression, capturing and stuff.
[00:12:50]
But you still have to know how to formulate a valid one that the regular expression will
[00:12:54]
capture and what it's going to do with that.
[00:12:57]
And to me, that that's not any easier than just writing high level instructions that
[00:13:02]
say, go to, you know, maybe you have to like learn what the pipe operator is, and then
[00:13:06]
you have to like learn a few of these things.
[00:13:08]
But then you just say, click this thing, do this thing, do this thing.
[00:13:12]
So you know, it's sort of like AppleScript went this route to of like, if we write it
[00:13:18]
like English, it will be easy for people to write, but it's actually not because it creates
[00:13:23]
this layer of abstraction that just makes it harder to understand what it's actually
[00:13:27]
going to do and how it's going to interpret it.
[00:13:29]
Kind of sounds like graphical codes, like, like dark, for instance, where you connect
[00:13:36]
things visually, like, yeah, but you have more rules, like, do you, do you know more
[00:13:41]
people who can write code that compiles or more people who can write English perfectly
[00:13:46]
grammatically?
[00:13:47]
Exactly.
[00:13:48]
That's going to be interpreted by a specific set of code instructions, and if it doesn't
[00:13:53]
fit that format, it won't do what's intended.
[00:13:55]
And how do you debug that?
[00:13:56]
So it's just adding a layer of indirection.
[00:13:58]
So anyway, for what it's worth, I'm very happy with just the high level way of expressing
[00:14:04]
things with Elm program tests.
[00:14:05]
I think it's, I think it's really good.
[00:14:07]
And I really enjoy being able to like, you know, write tests from the user's perspective.
[00:14:13]
I know a lot of French developers who would be better off writing code than English.
[00:14:19]
So well, so to just one thought on top of that is that I think in practice, especially
[00:14:26]
if you have a large application and you start using Elm program test that ideally you'll
[00:14:32]
end up with a module or maybe some different modules of helper functions that build on
[00:14:38]
top of Elm program test, some application specific helpers like the login example that
[00:14:45]
we said, or maybe in the ed tech domain, creating an assignment, which might do a whole bunch
[00:14:51]
of steps like, Oh, click the whatever the button to go to the create assignment page,
[00:14:56]
fill out all this data, click Submit, simulate the response.
[00:15:00]
So you can still have some of those benefits of having even higher level concepts.
[00:15:05]
And Elm program test is kind of like a support layer that does all of the generic stuff,
[00:15:11]
basically all of the reusable things about writing high level tests on program tests
[00:15:16]
does so that as an app developer, you can write your app specific tests much more quickly
[00:15:21]
and efficiently and correctly, which I guess is a point we should get to at some at some
[00:15:26]
point of all the things that Elm program test does behind the scenes.
[00:15:31]
Right, right.
[00:15:32]
And to that point of doing it correctly, I think that that's a really key point that,
[00:15:37]
you know, having this encapsulated in this library, I'm sure you could, you know, call
[00:15:42]
the update function yourself from an Elm test, but you don't have the same level of confidence
[00:15:48]
that things are actually being wired up in a way that's equivalent to what's going to
[00:15:53]
happen when you do browser dot application or whatever type of program you're creating.
[00:15:58]
So there's a lot of value to having that encapsulated in something that you can trust to be equivalent
[00:16:04]
so that you know you're not you're testing something that's realistic and going to reflect
[00:16:08]
the reality.
[00:16:09]
Yeah.
[00:16:10]
And to hit on that, just a little bit more about the types of refactorings that these
[00:16:14]
programs written in Elm program tests can help you refactor.
[00:16:18]
These are things like, oh, you have this set of messages and you want to change what the
[00:16:23]
messages are so that you can whatever centralize your logic, maybe in a certain way in your
[00:16:28]
update function and extract some data type specific functions that are then used rather
[00:16:34]
than having all that logic in your update function.
[00:16:37]
That's the type of refactoring that is very tedious to do if you don't have these high
[00:16:41]
level tests, because essentially what you have is a bunch of unit tests calling an update
[00:16:46]
function that are written in the language of that module that knows about its messages,
[00:16:52]
knows about its internal model.
[00:16:54]
And now you want all those unit tests to move to a different module that's specific to this
[00:16:59]
smaller data type doesn't know anything about messages.
[00:17:03]
So if you have the high level test coverage, you have some safety.
[00:17:06]
And if you don't have these higher level tests, you basically have to translate all your tests
[00:17:11]
and move them over, which is, I don't know, that's that's one of the things I I fear the
[00:17:16]
most when I'm doing coding.
[00:17:19]
Right.
[00:17:20]
And one example is like form validation is actually something fairly common and also
[00:17:24]
something that you tend to do incompletely the first time you implement it like, oh,
[00:17:30]
we'll just need to validate a few things, we'll move on.
[00:17:33]
And then over time, you need to add more validations, maybe at some point, you end up extracting
[00:17:37]
a validation helper module or using some package to help you with validation.
[00:17:43]
Those again, are things that can completely change the flow of events.
[00:17:48]
Like maybe you used to validate things on when the message happened, but now you're
[00:17:53]
switching to store the unvalidated data in your model and validating it when you send
[00:17:57]
the form and showing error messages in a different way.
[00:18:00]
Those are changes that ideally should be simple to make.
[00:18:04]
But if you're writing unit tests, they become extremely tedious, because those unit tests
[00:18:10]
that touch your update function or directly refer to messages are extremely brittle to
[00:18:15]
those kind of changes.
[00:18:16]
Right.
[00:18:17]
So if you were doing something like an optimistic update in the UI where you're interacting
[00:18:22]
with server responses, you would have to fill in a lot of the pieces to simulate that in
[00:18:28]
a plain old unit test.
[00:18:29]
But if you're driving it through on program tests, then you can say, click this button,
[00:18:34]
you know, enter this information, hit send, simulate a server response and make assertions
[00:18:40]
about the view while it's loading before the server responses come back.
[00:18:44]
So you can you can decouple it and and not rely on wiring it in a very specific way that
[00:18:50]
you can't trust as much.
[00:18:52]
So yeah, there's a lot of value to that.
[00:18:54]
Yep.
[00:18:55]
One thing I don't think we wrap this up yet, but you had mentioned earlier, Dillon, about
[00:18:59]
using unit tests to cover all the different edge cases and combinations.
[00:19:04]
And I would definitely agree with that.
[00:19:07]
I tend to start with maybe like a happy path program test.
[00:19:12]
Then you can jump down and do unit tests for all your edge cases.
[00:19:15]
That's cool.
[00:19:16]
So you like to do like a more outside in approach?
[00:19:19]
Yeah, yeah, exactly.
[00:19:21]
That's cool.
[00:19:22]
Because that so that's a really interesting topic that, you know, in the sort of test
[00:19:26]
driven development world, there, there are a lot of conversations about do you do you
[00:19:31]
write a unit test first, and then work your way up to building to fitting that unit into
[00:19:39]
the application, which is in a way, you know, that's delayed integration, you haven't fit
[00:19:44]
the piece into the hole.
[00:19:46]
So you spend time building up all these different cases and then fit it into the hole.
[00:19:51]
And you don't know if it's actually going going to solve the problem that you set out
[00:19:55]
to solve when you built that unit.
[00:19:57]
So you don't know if it's going to fit fitting fitting and integrating the piece into the
[00:20:00]
hole is the hard and risky part.
[00:20:02]
So the sooner you can do that, the better the outside in school of thought is more that
[00:20:06]
start with building something end to end.
[00:20:10]
And it starts with testing from the outside in from the user's perspective, and then builds
[00:20:15]
the unit as needed.
[00:20:16]
But it sort of uses more of a fake it till you make it approach, like you said, sort
[00:20:21]
of getting that happy path.
[00:20:22]
So that's more the approach that you like to take.
[00:20:24]
Well, that's interesting.
[00:20:25]
I don't know that I have a specific preference for either of those in Elm.
[00:20:31]
But I do tend to kind of write the high level test first, and then often I'll comment it
[00:20:37]
out or stash it or something and build those smaller pieces, then bring back the failing
[00:20:43]
high level test and plug the pieces in.
[00:20:45]
I think I would tend to do that most of the time.
[00:20:49]
However, there are cases if I'm not sure how things will be structured that I would write
[00:20:53]
the failing high level test implemented, like do the fake it till you make it and just write
[00:20:58]
a really simple thing where it's like hard coded to always show the stuff that's needed
[00:21:02]
to make it pass refactor.
[00:21:04]
I think there is a pitfall in doing that approach with on program tests that you can often skip
[00:21:10]
the refactoring step, or you can end up with working stuff that you've refactored, but
[00:21:16]
you haven't exactly extracted the coherent modules yet.
[00:21:20]
And you can end up with kind of a mess relying on only on program test if you aren't disciplined
[00:21:26]
about looking for the smaller pieces that you're going to pull out and write lower level
[00:21:31]
tests for those pieces.
[00:21:33]
Because when you use a regular Elm test, you tested an API and then by using that API,
[00:21:42]
you can kind of feel the pains with it.
[00:21:44]
But with Elm program tests, you don't feel those pains because those APIs are technical
[00:21:50]
detail and you can forget it.
[00:21:53]
Is that what you meant?
[00:21:54]
That they are a technical detail.
[00:21:56]
Yeah, sort of.
[00:21:57]
I think that tell me if this is related to what you're saying, like in Elm in I think
[00:22:03]
the preferred way and the way people are settling on is that if you have smaller pieces of your
[00:22:09]
own program, especially if they're related to view stuff or update logic, there's a lot
[00:22:14]
of different patterns you can use.
[00:22:16]
Like you may have a module that just has a function that returns HTML, or you might have
[00:22:20]
one that has a full update function that can produce commands or somewhere in between where
[00:22:25]
maybe it returns like some specific set of things that can result from its actions or
[00:22:31]
special functions to process the messages it can produce.
[00:22:35]
I think like that's a case where Elm program tests can be used to write unit tests for
[00:22:41]
modules like that that are not completely programs in the full architecture sense of
[00:22:47]
the word, but are also more complicated than just some functions.
[00:22:52]
However, Elm program test doesn't let you interact directly with its API.
[00:22:58]
So in like, and as you two talked about in your previous episode, writing tests in in
[00:23:03]
test driven development, or test driven design, I almost said, it's like a design tool.
[00:23:10]
And I think Elm program test doesn't really give you those design tool benefits, because
[00:23:15]
yeah, I think you're not working directly with the API, but it does help you.
[00:23:21]
It gives you other benefits, the test coverage we talked about that lets you do bigger refactoring
[00:23:25]
safely.
[00:23:26]
And it also helps you think about the user perspective more if you're really doing user
[00:23:32]
centered design.
[00:23:33]
And a process like that, it can really help get developers aligned with the kind of product
[00:23:38]
level thinking as well.
[00:23:40]
But for designing an API, Elm program test really doesn't provide those benefits of testing.
[00:23:46]
Right, it's almost like the, you know, user interface based testing in Elm program test
[00:23:54]
keeps you honest about making sure that you're designing something that's going to have a
[00:23:58]
nice experience for from the user's perspective.
[00:24:01]
And doing unit level tests with Elm test helps keep you honest that the API you've designed
[00:24:08]
is going to be nice to work with because that's the direct thing that you're exercising and
[00:24:13]
getting feedback on.
[00:24:14]
So you sort of want a mix of those high level and low level tests to make sure those things
[00:24:19]
are both being designed nicely.
[00:24:22]
It's really nice that you can write both using Elm tests that you don't have to write unit
[00:24:27]
tests in Elm and integration tests using JavaScript, using Cypress, for instance.
[00:24:34]
Even though that's an amazing tool, but that's a different thing.
[00:24:37]
Yeah, which is really one of the it's really leveraging the strength of Elm in that everything
[00:24:43]
in Elm is pure functions.
[00:24:45]
There's a virtual DOM, which is essentially a data structure.
[00:24:48]
The commands and subscriptions are essentially just data that represents what the runtime
[00:24:54]
is going to do.
[00:24:55]
So program test essentially is simulating or like reimplementing in Elm in kind of a
[00:25:03]
test specific way, the runtime of Elm where it's creating a model, it's, it's maintaining
[00:25:09]
it, if there's external effects, it's like keeping track of those.
[00:25:12]
Remembering what things are in progress, what are waiting for responses, and can render
[00:25:18]
the view from the current state at any time.
[00:25:21]
So yeah, it's all of that is possible and like was relatively easy to implement.
[00:25:26]
I mean, it's a lot of work, but it, it was easy to implement compared to other language.
[00:25:32]
Like actually, I've worked on similar test frameworks for Android, the Robo electric
[00:25:36]
framework at Pivotal Labs, we started writing a similar framework for iOS, which I think
[00:25:42]
never got finished.
[00:25:43]
I've like done similar things in the past and it's like, this is by far the easiest
[00:25:48]
to write in the most sophisticated of any similar attempts I've worked on.
[00:25:52]
That's cool.
[00:25:53]
It's like a, it's like a pure Elm implementation of simulating the Elm architecture and the
[00:25:59]
Elm runtime.
[00:26:00]
Yep, exactly.
[00:26:02]
So let's, let's get into a couple of things.
[00:26:05]
So first of all, I think we, we should, we should make it clear we've sort of implied
[00:26:11]
this but not said it explicitly.
[00:26:13]
You can simulate HTTP requests with Elm program tests.
[00:26:16]
And that's one of the really killer features because you know, you would be hard pressed
[00:26:21]
to find, find a way to do that if you were sort of rolling your own thing.
[00:26:25]
It's really something you're, you're going to have to do a lot of work to get that same
[00:26:30]
result otherwise with that Elm program test.
[00:26:33]
So there's an API where you can sort of expect a certain HTTP requests to be performed.
[00:26:39]
You can say simulate responding with this HTTP response with this status code, and then
[00:26:45]
continue to make assertions in your, in your view after that.
[00:26:51]
Yeah.
[00:26:52]
So just to compare what that would look like without Elm program test, you could do things
[00:26:57]
like that.
[00:26:58]
But you would in the first place be like saying call update with this exact message with all
[00:27:04]
this data.
[00:27:05]
Right.
[00:27:06]
And then you would just add it to the specific message type for your application.
[00:27:10]
Yes, exactly.
[00:27:11]
And you also wouldn't buy like, it starts to get extremely tedious going that manual
[00:27:17]
route, which is what I started with before I extracted on program test is that, okay,
[00:27:23]
well, maybe you also want coverage to make sure that the message that produced the HTTP
[00:27:29]
requests happened, because like, maybe you whatever disconnect the code that makes the
[00:27:34]
button appear that lets people initiate the request, or maybe the validation failed, and
[00:27:39]
the request wasn't even sent, things like that, Elm program test handles for you.
[00:27:45]
And then also, if your test fails, having a nice error message about what is happening
[00:27:51]
is another part that is a lot of tedious work to do on your own, which Elm program test
[00:27:57]
provides for you.
[00:27:58]
So if you say, Oh, simulate the response to this endpoint, and it was a timeout error,
[00:28:04]
if that request wasn't made on program test will say, Oh, that request was never made.
[00:28:09]
Here's the list of requests that were made.
[00:28:11]
In contrast, if you were just doing it manually sending the message, there's no guarantee
[00:28:15]
that like, maybe the message was sent, maybe it wasn't, you haven't validated it, validated
[00:28:20]
it all you know is like, maybe there was a typo in the URL, and it didn't match for that
[00:28:25]
reason.
[00:28:26]
So that's, I guess, something I take pride in, in Elm program test.
[00:28:30]
It's like, yeah, it's, it's a centralized place where and hopefully other people can
[00:28:34]
contribute to this to to have a nice error messages for these cases that like, ideally,
[00:28:39]
you want to have this, but you'd never have time to do that work to do that on your own,
[00:28:44]
if you didn't have this reusable package,
[00:28:47]
right?
[00:28:48]
Instead of everybody individually building their own thing and taking their time to do
[00:28:52]
that, hopefully, you know, you've, you've done the bulk of the work, but hopefully others
[00:28:57]
can then invest some of that time they would have spent rolling their own thing to improving
[00:29:01]
this community resource.
[00:29:03]
Yeah, so there's a whole bunch of stuff.
[00:29:05]
So on program test, like you mentioned, can do simulations of HTTP responses, it can partially
[00:29:12]
do the simulation of time passing, like if you have tasks that have delays, it can simulate
[00:29:19]
that and you say like, Oh, advanced time by this much, and it'll trigger all the all the
[00:29:24]
delayed tasks, you can simulate ports, both incoming and outgoing ports, and you can simulate
[00:29:30]
some of the browser and Dom API, that's an area I still need to improve quite a bit.
[00:29:37]
So what is it that you can't simulate with Elm program test at the moment?
[00:29:41]
Let's see.
[00:29:42]
So specific things like browser focus, like focusing on particular IDs in the DOM viewport
[00:29:49]
scrolling is something that I haven't touched on.
[00:29:52]
There's some things related to time, I think, like the subscriptions for time, like time
[00:29:56]
dot every aren't implemented, but I would like to implement.
[00:30:01]
I think that's, that's the basics of it.
[00:30:03]
I don't know if there's any other kind of niche packages, I guess, like the file and
[00:30:07]
bytes API is something that I haven't looked at yet.
[00:30:11]
But those are things that like, kind of the internals are set up where a lot of that stuff
[00:30:16]
could be implemented.
[00:30:17]
You just have to think about, okay, what's the API of different error conditions that
[00:30:22]
people may want to simulate in tests?
[00:30:24]
What's the internal model that would represent this?
[00:30:26]
What do we want the error messages to look like?
[00:30:29]
So those are definitely things I'd love to have contributions for if someone needs those
[00:30:34]
features for testing whatever their application is.
[00:30:38]
So there's nothing really blocking those things from existing.
[00:30:42]
It's just more time and money, or more time.
[00:30:46]
Yeah, it's free.
[00:30:49]
I'd say there's one, like some of the things like, okay, keyboard focus is something that
[00:30:56]
ideally you'd want to be able to simulate, but the logic that browsers use to maintain
[00:31:02]
that is so complicated.
[00:31:03]
Yeah, I don't know if that could ever be safely done.
[00:31:07]
Maybe I like that's something I'd love to be able to simulate here.
[00:31:11]
But it's also of like, limited use.
[00:31:13]
Like that's the point where maybe you just have to track things manually and say, okay,
[00:31:18]
well, here's the state of the DOM.
[00:31:20]
Here's what's going to happen.
[00:31:21]
Let's like hack together a test that gets things into that state.
[00:31:24]
Yeah.
[00:31:25]
Or there at some points, you should probably just use a better tool, like Cypress, a tool
[00:31:30]
more suited to the task.
[00:31:32]
Yeah, I mean, Elm program test is not, it seems like Elm program test is not intending
[00:31:37]
to be something that is actually performing HTTP requests, actually sending ports and
[00:31:44]
executing JavaScript.
[00:31:45]
And that's like, you know, by design, which, so sometimes I, you know, it's, it gets a
[00:31:51]
little confusing, but people use different terms for this.
[00:31:54]
I think of it as like end to end testing versus integration testing, where, you know, I would,
[00:31:59]
I would consider Elm program test to be an integration testing framework.
[00:32:03]
It's not ever going to make an HTTP request, which is a feature or a bug depending on what
[00:32:11]
you're trying to do, right?
[00:32:12]
It is, you have to pick the appropriate testing level and you have to keep that in mind, but
[00:32:19]
obviously it's going to be faster.
[00:32:21]
It's going to be more deterministic if you're not actually making HTTP requests, right?
[00:32:26]
So you need to have an awareness of what the tool is suited for and use it for those effects.
[00:32:33]
So, so like I want, I wanted to make the API a little more concrete for people maybe.
[00:32:40]
So like when it comes to like simulating HTTP effects, so essentially you have this sort
[00:32:46]
of program test data type.
[00:32:49]
If you think about, if you think about the Elm language, it is not a dynamic language.
[00:32:53]
It is a static language.
[00:32:55]
It's not a language where you can go in and tweak the internals of something, reach in
[00:33:00]
and change global variables or, or that sort of thing.
[00:33:05]
So everything needs to be sort of injected rather than reached in and modified.
[00:33:10]
If it was a language like Ruby or JavaScript, a lot of these frameworks work by, you know,
[00:33:15]
monkey patching, you know, in Ruby you can like override existing classes and actually
[00:33:21]
reach in and modify them.
[00:33:23]
Elm doesn't work that way.
[00:33:24]
Monkey patching is not a thing in Elm language.
[00:33:26]
And so, but everything is pure functions.
[00:33:30]
And so the way that that works in, in Elm is you, you pass, you pass things in explicitly
[00:33:36]
and effects are actually just a type of data.
[00:33:40]
And so when you're simulating HTTP effects, what you do is you have this program test.
[00:33:47]
So an Elm, an Elm test case is just, it's just a single expectation ultimately under
[00:33:55]
the hood.
[00:33:56]
And you know, it's just this one expectation type and a program test sort of builds up
[00:34:01]
this program test and you sort of chain on a sequence of things.
[00:34:06]
So it's inherently a sort of imperative flow that you're describing.
[00:34:11]
The user goes in and does this and does this and does this, but you're actually building
[00:34:15]
up a single expectation that describes that sequence of events.
[00:34:20]
So it's a single pipeline and you can chain on.
[00:34:23]
So you sort of initialize your program test and give it your update function, your view
[00:34:28]
function, your init function, all that it needs to sort of create its mini simulation
[00:34:34]
of the Elm runtime.
[00:34:36]
And then you can simulate HTTP requests and say you expect to get a particular HTTP request
[00:34:44]
and it's going to essentially mock that out.
[00:34:46]
Yeah.
[00:34:47]
Let me just like, as Elm programmers, we like to think about the data types.
[00:34:51]
So actually this program test type is pretty straightforward.
[00:34:56]
You can think of it as a result.
[00:34:58]
It's like either in an error state or in a success state.
[00:35:02]
In the error state, there's kind of a whole different set of different kinds of errors.
[00:35:06]
Like it's maybe we failed because we expected an HTTP request.
[00:35:11]
Here's the function that you were trying to call.
[00:35:14]
Here's this, here's the data of other requests that were in flight at the time.
[00:35:19]
Basically whatever's needed to produce a nice error message.
[00:35:21]
And in the success case, we've got the current model of your program.
[00:35:26]
And then we have some other state about the world of like what HTTP requests are in flight,
[00:35:32]
what delayed effects are you like scheduled in time?
[00:35:38]
What ports exist?
[00:35:39]
What outstanding like browser requests have been triggered?
[00:35:43]
So it's, it's really just a data structure like that.
[00:35:46]
And then the functions that you call like click button, all it does is it's going to
[00:35:50]
render the view based on the current state of the model.
[00:35:53]
Look for the button that you wanted to click, grab the message from that, call the update
[00:35:58]
function.
[00:35:59]
So I guess the update and view functions are also stored in program test.
[00:36:03]
But yeah, if you like a lot of it, you can kind of understand if you think about that
[00:36:07]
as the data structure of what's in there.
[00:36:09]
It's just a result with your program state, the functions for your program and information
[00:36:14]
about external effects that are in flight.
[00:36:17]
Right.
[00:36:18]
And there's a, there's one convention that maybe is like a little tidbit that might be
[00:36:23]
helpful to talk about for people.
[00:36:25]
There's a convention of like these insure function calls and expect function calls.
[00:36:31]
So you can insure that an HTTP request was made.
[00:36:35]
So there's program test dot insure HTTP request, and then there's program test dot expect HTTP
[00:36:41]
request.
[00:36:42]
You want to talk about that distinction?
[00:36:44]
This is one of the rough edges with the API here, because there's a school of testing
[00:36:50]
thought where you should have one expectation per test, and that should be the end of your
[00:36:55]
test.
[00:36:56]
So those are the primary functions here, the functions that take your program test value
[00:37:01]
and return an expectation that basically ends your test.
[00:37:05]
However, when you're writing these high level things, a lot of times you want these intermediate
[00:37:09]
assertions.
[00:37:10]
Like, oh, your first step in your test is to have the user log in.
[00:37:14]
Well, maybe you're not on a page that has the login form, so you need to fail there
[00:37:18]
with an error message.
[00:37:20]
So there's basically a copy of all the expect functions have an insure function that basically
[00:37:26]
does the same thing, but it returns a program test so that you can do an intermediate assertion,
[00:37:32]
continue your test and get to the end.
[00:37:36]
I wish there was a way to unify those into a single check, but I haven't come up with
[00:37:41]
a good API design approach to do that.
[00:37:44]
It almost seems like it could be like a builder pattern where you just only have either, you
[00:37:50]
know, pick a keyword, insure or expect, but only have the current behavior of insure,
[00:37:56]
which means that you can keep chaining things on.
[00:37:58]
And then at the end, you say to expectation, and that takes the whole test scenario and
[00:38:05]
turns it into a single expectation, which is the same as currently what expect HTTP
[00:38:10]
request does versus insure.
[00:38:12]
Yeah.
[00:38:13]
So you can write your tests in that way.
[00:38:15]
There's a program test.done, which is the final step.
[00:38:20]
But I didn't want that to be the recommended approach because I think the thought of like
[00:38:26]
having a single expectation is, in my opinion, the preferred way to think about your tests.
[00:38:32]
So I didn't want to like make that recommended way be the second class citizen in terms of
[00:38:38]
how the API was designed.
[00:38:40]
Gotcha.
[00:38:41]
But then, so like if you wanted to, oh, okay.
[00:38:44]
So there are also like simulate forms.
[00:38:46]
Okay.
[00:38:47]
So, so there's like insure HTTP request and that's sort of making an expectation and continuing
[00:38:54]
on kind of what you're recommending against.
[00:38:56]
You're saying maybe this is a smell and if you have too many insurers, maybe think about
[00:39:02]
the way you're splitting up your test cases.
[00:39:04]
If you're inserting too much.
[00:39:06]
Yeah, I'm kind of on the fence about that because I think like we were saying earlier,
[00:39:11]
you don't want a ton of these high level tests to check every edge case.
[00:39:15]
You kind of want to go through your happy path, which is going to do a lot of things.
[00:39:19]
So you tend to want to assert things along the way.
[00:39:24]
Having intermediate assertions also helps you with debugging when something goes wrong.
[00:39:28]
You can kind of tell what step failed.
[00:39:30]
So yeah, that's something I wish I had a better answer for that.
[00:39:36]
But unfortunately you kind of learn that thing that's specific to on program tests.
[00:39:41]
If you have expect and insure, they're basically the same thing that you use throughout your
[00:39:45]
tests.
[00:39:46]
Yeah.
[00:39:47]
I mean, in a way there are implicit expectations in certain things.
[00:39:52]
If you say click button with text, then you're asserting that there's a button with that
[00:39:57]
text and it will fail, right?
[00:39:59]
Yes, exactly.
[00:40:00]
You don't have to say expect.
[00:40:02]
So that in your book wouldn't count as like too many assertions in the test.
[00:40:07]
It's just...
[00:40:08]
In fact, yeah, those are things where on program tests is helping you get a nicer error message
[00:40:14]
when something goes wrong.
[00:40:15]
So actually like I think you could probably realistically use a rule saying that your
[00:40:21]
normal tests should never use the insure functions.
[00:40:24]
But if you're writing application specific helper functions for your tests, like you're
[00:40:29]
writing a helper function that's like to the login form, you can use insure functions in
[00:40:34]
that helper function, but then your actual test will just call that login thing.
[00:40:39]
It's essentially a higher level version of the click button type helper.
[00:40:43]
And then your actual test only has expect at the end.
[00:40:48]
That's very cool.
[00:40:49]
Tell me if you want to work on that rule with me.
[00:40:51]
Yeah.
[00:40:52]
Because I could be on review rule.
[00:40:54]
That sounds great.
[00:40:55]
Yeah, so just to reiterate what you're saying here, like when you say click text and we're
[00:41:01]
saying there's an inherent expectation and clicking the text because you're asserting
[00:41:07]
that that text exists on the page, that you're saying that you could use, if that didn't
[00:41:12]
exist as a building block, you could create that your own building block like that where
[00:41:16]
you say log user in.
[00:41:18]
And if there's no login button, you could create the validation that that exists in
[00:41:23]
your own helpers and use insure in that context.
[00:41:26]
Yep, exactly.
[00:41:28]
And for instance, the the API of own program test actually enforces certain best practices
[00:41:35]
like it has a click button function that you can call, but it can only click things that
[00:41:41]
are actually buttons or that are divs that have accessibility attributes that indicate
[00:41:46]
their buttons.
[00:41:48]
So if for whatever reason, your program is just a bunch of divs with on click handlers,
[00:41:53]
the built in click button function in on program test is not going to work.
[00:41:58]
I could make it work, but I've chosen not to.
[00:42:01]
But you can always write your own helper function that uses the like, find DOM element and do
[00:42:07]
this with it to kind of build your own helper functions that are needed to do whatever you
[00:42:13]
want that your app needs.
[00:42:14]
Yeah, that's an amazing feature.
[00:42:16]
So I guess we should maybe talk about some more of the rough edges.
[00:42:20]
Sure.
[00:42:21]
So program test, I think the big one, which is kind of the most unfortunately, the most
[00:42:26]
complicated thing is if you're if you're testing a program where you want to test these external
[00:42:32]
effects and simulate HTTP responses, or simulate interacting with your JavaScript ports, things
[00:42:40]
like that, there's a limitation in in Elm at the moment where there's these data types
[00:42:45]
command and subscription, or sub that in theory, like, theoretically, those things just are
[00:42:52]
a piece of data that represents some effect that the on runtime is going to perform on
[00:42:56]
your behalf.
[00:42:57]
So in on program test, I need to take a command that your program produces and know was there
[00:43:03]
an HTTP request made in that command and all the various different commands.
[00:43:09]
So unfortunately, that's not possible in Elm at the moment.
[00:43:12]
So well, okay, so I know that Evan has had a concern.
[00:43:17]
And this is a similar reason to why the HTML type is not directly inspectable or destructurable
[00:43:25]
is that Evan's been conservative about allowing for those kinds of destructurings in an attempt
[00:43:30]
to prevent packages getting published that do extremely complicated or like not explicit
[00:43:38]
things.
[00:43:39]
So I think it's about the explicit as of Elm.
[00:43:41]
So there's no internal technical reason they couldn't be inspectable.
[00:43:46]
And I think it's something that like, it's not even a decision that they should never
[00:43:50]
be inspectable.
[00:43:51]
But it's that in production code, I think specifically, Evan has wanted to avoid allowing
[00:43:56]
the use of packages that can like take a command and transform it into some other command,
[00:44:02]
which could allow for things like oh, sending all of your HTTP requests, if you're using
[00:44:06]
this caching package, it transforms all your HTTP requests to go to some other server and
[00:44:12]
redirect all your private information to a man in the middle attack or something like
[00:44:16]
that.
[00:44:17]
Right.
[00:44:18]
Yeah.
[00:44:19]
So I think like, my understanding is it's something that Evan would be open to and does
[00:44:24]
make sense, but only in the context of testing, and he would not want an API that's usable
[00:44:29]
in production code to be able to inspect at that level.
[00:44:33]
And you couldn't do a test that says this HTTP request, so HTTP.get with a URL, and
[00:44:42]
then say expect equals to something else?
[00:44:46]
In general, no, because so HTTP specifically, there are functions in that HTTP request type.
[00:44:54]
And most commands in fact, do have fun, like even your port commands, which are the simplest
[00:44:59]
ones, there's a function that takes the port value and turns it into a message.
[00:45:03]
Yeah, functions pretty much everywhere.
[00:45:05]
I guess functions can be equivalent by reference, but it gets to the area where it's like not
[00:45:12]
entirely reliable.
[00:45:14]
I believe Elm will, if you say function one double equals function two, if it's the exact
[00:45:21]
same reference, then it will equal true.
[00:45:24]
And otherwise, it will crash the program.
[00:45:26]
It'll give a runtime exception.
[00:45:28]
Yeah.
[00:45:29]
I think that's one of the things that they want to change in Elm 0.20.
[00:45:35]
Not getting those crashes when you compare functions or JSON regexes.
[00:45:39]
Yeah, maybe.
[00:45:40]
Well, the regex one has been fixed, but I think the idea maybe would be to somehow disallow
[00:45:47]
comparison of functions or something.
[00:45:50]
So we should talk a little bit about the effect pattern and how it relates to Elm program
[00:45:55]
test.
[00:45:56]
Yeah, that's where we're going with that is to work around that is that I've had to, you
[00:46:01]
basically have to refactor your program first to define your own data type that represents
[00:46:06]
the effects you're going to produce.
[00:46:08]
Then you have in production, a function that turns those effects into commands.
[00:46:13]
And in the test side, you have basically a parallel function that does the exact same
[00:46:18]
thing but it just returns a different type, which is this simulated effect type that's
[00:46:22]
specific to Elm program test.
[00:46:25]
So like, HTTP dot get is not inspectable.
[00:46:29]
But if you had your own, you know, version of simulating an effect, so you have, like,
[00:46:36]
you click a button and it says, you know, that button fetches the latest to do items.
[00:46:41]
So you have like an effect fetch to dos.
[00:46:45]
So instead of that just being a command, HTTP dot get slash to dos dot JSON or something,
[00:46:51]
it's going to be an effect called fetch to dos.
[00:46:55]
Your update function is going to be wrapped in a little helper that is actually your the
[00:47:02]
direct update function you you write is going to be returning a model comma effect.
[00:47:09]
And then you're going to have to translate that effect in your main production code into
[00:47:15]
HTTP dot get slash to dos for the fetch to dos effects type.
[00:47:20]
So it's just a custom type type effects equals fetch to dos and then all the other possible
[00:47:25]
effects.
[00:47:26]
But then in the test one, you write a translator that turns that effects not into HTTP dot
[00:47:31]
get but into simulated effects dot HTTP dot get.
[00:47:37]
So it's a drop in replacement except instead of importing HTTP, you're importing simulated
[00:47:43]
effect dot HTTP as HTTP.
[00:47:46]
Otherwise it's the same API, but it gives you something that you can inspect in on program
[00:47:51]
test.
[00:47:52]
Yep.
[00:47:53]
And unfortunately, the HTTP package has a similar restriction where you can't inspect
[00:48:00]
the HTTP body type or the HTTP expect type that's used for parsing.
[00:48:07]
Maybe you can inspect body.
[00:48:09]
But anyway, there's kind of this chain of things where you can't if you have an HTTP
[00:48:14]
expect type that like represents the decoder and all of that, you can't actually use that
[00:48:19]
for anything directly.
[00:48:20]
So that's another thing that I have to have a parallel version that's used in the test.
[00:48:25]
So just to run through how I think about doing that in the least annoying way possible is
[00:48:33]
you want to you have to look at the function that you're calling in the real world that
[00:48:38]
is producing the command.
[00:48:40]
So in your example, HTTP dot get returns a command.
[00:48:44]
So your effect type should really just take the parameters to that.
[00:48:49]
So a little different than what you were saying you were saying you could have a fetch to
[00:48:53]
do is effect.
[00:48:55]
I think it's actually better to keep your effect type more generic where it's just representing
[00:49:00]
the functions that are going to get called.
[00:49:02]
So you'd have I'd recommend like it doesn't reflect the message type.
[00:49:07]
It's more akin to the command type, but it's an indirect a level of indirection.
[00:49:12]
So you can translate it into a simulated or a real command.
[00:49:15]
Yeah.
[00:49:16]
So like get takes the URL as a string.
[00:49:19]
It takes the HTTP expect, which again, we we have to fake.
[00:49:24]
So then to create your expect, there's like the JSON body or expect JSON or whatever.
[00:49:31]
So the parameters to the to that is what you'd stick in your effect type.
[00:49:36]
So then on the real side, you just call HTTP get with the parameters and build it up.
[00:49:42]
And then the simulated version should look exactly the same because there's a whole bunch
[00:49:46]
of like there's simulated effect that HTTP which is a module.
[00:49:50]
It has exactly the same API as the API as the HTTP module.
[00:49:55]
It just returns simulated effects instead of commands.
[00:49:59]
So it's basically a bunch of boilerplate.
[00:50:02]
That's pretty annoying and takes up a whole lot of documentation of Elm program.
[00:50:09]
But theoretically, it's possible to remove that limitation.
[00:50:12]
It's kind of like a detailed project.
[00:50:15]
If anyone out there is looking to help with this, ideally, on program tests shouldn't
[00:50:20]
need any of this, it should just be able to inspect your commands and read the data without
[00:50:25]
you having to do anything to your program to be able to start using it.
[00:50:29]
That would be a game changer.
[00:50:31]
Yeah, I've actually in Elm 18.
[00:50:34]
I prototyped a package that could do that and it needs kind of some integration with
[00:50:39]
the test runner itself to swap test only JavaScript to make that possible.
[00:50:45]
But ideally, it should happen.
[00:50:46]
It just hasn't had a chance to get implemented yet.
[00:50:49]
I'm wondering whether the effect pattern still has some merits if you're able to do that,
[00:50:55]
or if it's just uses boilerplates when we get to that point.
[00:50:58]
Yeah, I think like this is, it's a question that I've seen asked more on the Haskell side
[00:51:05]
of things.
[00:51:06]
Where in Haskell, there's a type called IO, which basically is similar to Elm's command,
[00:51:13]
where it's saying that this can have external effects of any type.
[00:51:17]
But then if you're really into the strong typing and limiting the scope, letting your
[00:51:22]
types limit the scope of what functions can do, then you look at saying I want a function
[00:51:26]
that like can't do everything, but it can it's allowed to talk to the database, let's
[00:51:31]
say and it can maybe it's allowed to send log information, but it can't do anything
[00:51:36]
else like it can't read and write from the terminal console.
[00:51:39]
It can't I don't know what other effects are I can't like sleep the computer or call the
[00:51:45]
halt command, whatever.
[00:51:47]
So in that environment, there's a lot of conversation about the best way to model that.
[00:51:51]
And this effect approach is one style of that where you can have a data type that represents
[00:51:57]
just the limited set of things that are allowed, you can end up with a whole bunch of different
[00:52:01]
types that represent different contexts that are allowed.
[00:52:04]
So you could do a similar thing in in which personally, I think has an occasional use
[00:52:11]
where you maybe have some complicated UI component that wants to like ask for certain things
[00:52:18]
like maybe it needs to trigger a focus event somewhere else in the DOM.
[00:52:24]
So it but you don't want to allow it to make HTTP requests.
[00:52:28]
So you could do something like that where that module defines a type of things that
[00:52:32]
the parent component should do on its behalf.
[00:52:35]
And that makes sense.
[00:52:36]
But I think as a general, like, I think it makes sense in specific cases where you have
[00:52:41]
a component that like very clearly needs that responsibility.
[00:52:45]
But as a general pattern, I think it just gets overly complicated.
[00:52:49]
And in Elm program test, I would avoid even using that pattern at all if it were possible
[00:52:55]
to do.
[00:52:56]
Yeah, that makes sense.
[00:52:57]
I mean, it could potentially have the same benefit of, you know, how type signatures
[00:53:03]
in Elm allow you to reason about what's going on.
[00:53:07]
Is this changing the whole model?
[00:53:10]
Is that even possible?
[00:53:11]
Or is it narrowed down into, oh, this can only change this one data type.
[00:53:15]
So if I'm looking at this one piece of the model changing, I can narrow my focus and
[00:53:20]
ignore these areas of code.
[00:53:22]
You know, you can use the effect pattern to do similar things.
[00:53:25]
But it's so, as you're saying, it's sort of so heavyweight to do that, that it might not
[00:53:31]
be worth the benefit because managed effects in Elm are already so cleanly isolated that
[00:53:38]
it's in a pretty good state as it is.
[00:53:40]
I've thought about using a pattern like this for a sort of plugin architecture for authors
[00:53:46]
creating packages for Elm pages, because if you're performing a static HTTP request or,
[00:53:53]
you know, things like that, if you set up a package that generates an RSS feed, it would
[00:53:58]
be nice to know, is it allowed to make HTTP requests or not and be able to explicitly
[00:54:03]
give it permission for what it can do.
[00:54:05]
There are other ways to achieve that effect, though.
[00:54:07]
Like you could just not perform any requests that it performs except a particular wrapped
[00:54:13]
type and you have to pass in the request it can perform.
[00:54:16]
So you could pass in a reference to HTTP.
[00:54:20]
And anyway, there are other ways to achieve that effect.
[00:54:22]
Yeah.
[00:54:23]
And one other use of that pattern is if you need to process those effects in different
[00:54:29]
ways in different contexts.
[00:54:30]
So an example would maybe you have some kind of like, admin tool that lets you like, configure
[00:54:37]
the forms.
[00:54:38]
And actually, we did this at No Red Ink, we had an assignment that students could do.
[00:54:44]
And normally it would like, you know, send data to the back end, here's the work they've
[00:54:48]
done, get data from other students.
[00:54:51]
But we also wanted a preview mode where teachers could play around with the assignment, and
[00:54:55]
also like simulate other students doing work that would like get sent to you and things
[00:55:01]
like that.
[00:55:02]
So that was a case where having an effect type as data was useful, because we can interpret
[00:55:07]
those effects in different ways in the real mode for the students doing the work, we would
[00:55:12]
send the HTTP requests in the preview mode, we have a different interpreter that basically
[00:55:17]
like uses a fake preview data structure.
[00:55:20]
And we can trigger different effects from like an extra panel of buttons for the teacher
[00:55:24]
to explore.
[00:55:25]
So that's that's a case where you need this pattern, but also a relatively rare one, like
[00:55:31]
occasionally, you'll be doing where you need that capability, but not often.
[00:55:35]
That's cool.
[00:55:36]
I can imagine like wanting to have like security audit log or something of all of the particular
[00:55:42]
effects that have been performed.
[00:55:43]
And so you could have a loggable chain of those things and be able to trace them.
[00:55:48]
That's that's a really interesting thing to think about.
[00:55:50]
So okay, so I want to I want to loop back and talk a little bit more about this thing
[00:55:55]
you brought up with the effect pattern with Elm program test where you're describing using
[00:56:01]
the effects type that you define as a sort of equivalent of a command.
[00:56:08]
So you've got like type effects equals and then you can have no effect, which would be
[00:56:12]
the equivalent of command dot none.
[00:56:15]
And then you can have sort of instead of get to do's that specific message to make that
[00:56:21]
HTTP request for getting to do's, you're saying you could have a more general one.
[00:56:26]
And so you can have like a get data effect that has that takes a URL as part of its payload
[00:56:35]
of that constructor and it takes a decoder.
[00:56:38]
So there's an example that we'll link to in the show notes from the Elm program test examples
[00:56:43]
folder that sort of shows effects using HTTP simulations.
[00:56:48]
But how do you avoid having type variables in your effect type or or do you just have
[00:56:54]
to bite the bullet and have a get data for every specific data type that gets returned?
[00:57:01]
Yeah, because right, you need different like in one case, maybe you have one request where
[00:57:06]
you have a decoder that decodes users and in another request, you have a decoder that
[00:57:12]
decodes account information or yeah, to do to do it.
[00:57:16]
Exactly.
[00:57:17]
Yeah, so I need to I need to make a good example.
[00:57:20]
The example we're like to is kind of written as the simple way to do things and straightforward.
[00:57:28]
So there's not a ton of discussion, because the HTTP API in Elm is unfortunately a bit
[00:57:35]
tedious to work with.
[00:57:37]
There's a whole bunch of different types and like if you're doing if you're making tasks
[00:57:42]
versus commands, there's different there's like the expect type, but then there's also
[00:57:47]
some other type that I'm forgetting the name of.
[00:57:51]
So anyway, there is a way to deal with that.
[00:57:53]
The way you end up doing it is the effect type would have for instance, like an HTTP
[00:58:00]
request constructor, right, it would have the payload that it need the headers have
[00:58:05]
the method and the way you end up doing it is you have a decoder that decodes messages.
[00:58:11]
And then you also have a separate function that takes an HTTP error and returns a message.
[00:58:18]
And then you also have the body that takes a JSON value.
[00:58:23]
So you basically like hidden all of your request specific types behind JSON and behind message.
[00:58:31]
So that tedious to call directly, but then you make a helper function that can that basically
[00:58:37]
does take the decoder of type a, the function of type a to message or alternatively the
[00:58:43]
function of result HTTP error a to message, and then it builds the actual parameters to
[00:58:50]
the effect based on those and it would be like composing the decoder with the function
[00:58:56]
that takes the a value and turns it into a message and ends up just storing the decoder
[00:59:00]
of message in the effect type.
[00:59:03]
So that's that's the trick you can use to essentially hide things that like in Haskell,
[00:59:07]
you could deal with that with rank two types and things like that.
[00:59:10]
But in now, you have to have a helper function that has those extra parameters, but then
[00:59:15]
collapses them and return something that doesn't care about those parameters anymore.
[00:59:19]
Mm hmm.
[00:59:20]
Cool.
[00:59:21]
Yeah, I have I have an example that I can link to.
[00:59:23]
I don't I don't think there's an example of this in the own program test examples at the
[00:59:26]
moment, but I've got an example that I can link to that uses that pattern that you just
[00:59:31]
described.
[00:59:32]
So yeah, that's sort of the conclusion I came to as well that you essentially the the trick
[00:59:37]
is that you're using instead of the specific types you're decoding to, you're decoding
[00:59:42]
everything to a JSON decode value, and then you're having a function that takes that JSON,
[00:59:48]
JSON decode value and turns it into a message.
[00:59:51]
Hard to wrap your brain around, but yeah.
[00:59:53]
Yeah.
[00:59:54]
And finally, it's like way more tedious than I would like because the HTTP API just has
[01:00:00]
a kind of convoluted way of dealing with errors, depending on your exact call.
[01:00:06]
So but again, this is something that would just completely go away once the work to like
[01:00:12]
directly work with commands under test is done.
[01:00:15]
So yeah, again, if anyone wants to help get you, it would be incredible, easier to finish
[01:00:20]
that work than to figure out all the stuff we were just talking about, about how to deal
[01:00:25]
with the current API in your tests.
[01:00:28]
So we've gone pretty in depth on the effect pattern, why that's needed.
[01:00:33]
We've talked about simulating effects and the APIs for that.
[01:00:38]
We've talked about the testing pyramid, when to use unit tests versus high level tests.
[01:00:43]
Let's sort of do a quick round of best practices and tips and tricks to, you know, using Elm
[01:00:50]
program test effectively.
[01:00:51]
Sure.
[01:00:52]
So one thing we didn't, we didn't touch on a lot, but kind of a huge amount of work is
[01:00:58]
behind the scenes in Elm program test is the types of error messages that can report about
[01:01:04]
your view.
[01:01:06]
So for instance, if you say click a button with this label, it will look for buttons
[01:01:12]
in a whole bunch of different ways.
[01:01:14]
Like it'll check for accessibility tags, it'll check for things that are actual buttons,
[01:01:18]
it'll check for a button that has an image with all text in it of that text.
[01:01:24]
So I don't know if this is a best practice so much as just the best practice is to use
[01:01:29]
Elm program test.
[01:01:31]
It can help encourage you to write code in an accessible way.
[01:01:36]
Like another thing it will do is verify that you have a label for your checkbox.
[01:01:41]
If you want to click a checkbox and make sure that the label is hooked up in a way that
[01:01:45]
actually works, which there's like three different ways to do it, but you can also make mistakes
[01:01:50]
and have it not actually work in the browser.
[01:01:53]
Yeah, no, those are great tips.
[01:01:55]
I mean, I think those are good debugging tips for just how to understand if things aren't
[01:02:01]
wired up, what should you be learning more about or double checking that you've done
[01:02:05]
correctly?
[01:02:06]
Do you have any opinion on how to select tags or elements?
[01:02:11]
So you say, select them by label, select them by text, but I know that a lot of people like
[01:02:17]
to use end to end tests ID.
[01:02:20]
So a specific attribute, for instance, at Humio, we call it end to end dash ID, and
[01:02:27]
we only use those for tests.
[01:02:30]
Yeah.
[01:02:31]
So I think like the goal of Elm program test is to avoid needing those things in a lot
[01:02:37]
of cases, specifically like with buttons.
[01:02:40]
Elm program test is smart enough to be able to search for the labels.
[01:02:44]
And there's like five different ways a button could get label text.
[01:02:47]
And Elm program test does that in a reusable way.
[01:02:51]
So I think often a lot of the reason that people add those IDs is because the testing
[01:02:56]
framework they're using isn't smart enough to find the thing by the label, all the different
[01:03:02]
ways that labels can be attached.
[01:03:04]
So it's just easier to start attaching IDs everywhere.
[01:03:07]
So if you're using Elm program test, I think a lot of those cases, you won't need those
[01:03:12]
IDs because Elm program test is smart enough to find what you mean by some user viewable
[01:03:20]
information that's on the page.
[01:03:21]
However, there are some limitations to that.
[01:03:24]
Yeah.
[01:03:25]
I think one reason that I heard also is that people don't want to have their test fail
[01:03:31]
when the text of the button changes.
[01:03:33]
You consider that as part of the spec from what I'm hearing?
[01:03:38]
Yeah.
[01:03:39]
I mean, I would say the approach I'd recommend in Elm program test if you wanted that type
[01:03:44]
of safety is to define some constants somewhere that have the text.
[01:03:49]
And then you can refer to that both in your test and in the real code.
[01:03:54]
Right.
[01:03:55]
Because you are sharing code between your...
[01:03:57]
It's just Elm code between the tests and the code.
[01:03:59]
Yeah.
[01:04:00]
I'd say like that, the reason for that is because of my goal of wanting the tests to
[01:04:07]
read like something that a user could understand, or maybe someone understands what HTTP requests
[01:04:14]
are, but they can read it and see like, oh yeah, we're clicking the go back button or
[01:04:19]
whatever.
[01:04:20]
So that if you look at it, basically the scenario that in the workflow that you're testing is
[01:04:26]
clearly visible and readable as opposed to being hidden behind button IDs.
[01:04:32]
However, like if someone really wanted to have an ID focus thing, they could make a
[01:04:37]
test module of helper functions that do that.
[01:04:40]
Like there's some lower level things in Elm program test where you could implement the
[01:04:44]
set of helper functions that you want for your application if the ones that Elm program
[01:04:50]
test provides do some preferences that you don't really want.
[01:04:54]
But I think in general, like I'd like to see more focus on thinking from the user perspective
[01:04:58]
and haven't in practice seen a lot of issues with text labels changing where it was hard
[01:05:04]
to replace.
[01:05:05]
You just go and change it or extract a variable in a constant if that's something that you
[01:05:09]
have happening a lot.
[01:05:11]
So for the, we talked about these sort of effect handlers that are taking your custom
[01:05:16]
type representing the effects in your application and turning it into a real command and a simulated
[01:05:22]
command or a simulated effect.
[01:05:25]
How do we keep those in sync?
[01:05:28]
Because so like one thing that I think can be helpful is, you know, kind of like I hinted
[01:05:33]
at earlier, import simulated effect.http as HTTP.
[01:05:38]
If you import it with that import alias, then the handler function in your test code and
[01:05:45]
your production code will look exactly the same.
[01:05:48]
Yes.
[01:05:49]
Yeah, that's exactly right.
[01:05:50]
And all the simulated effect dot whatever modules in program test, if you look in the
[01:05:55]
docs, it says, this is meant to be an exact parallel of this real module.
[01:06:01]
So that's really the key as we talked about briefly earlier, the constructors in your
[01:06:07]
custom effect type should basically, like, ideally just be the parameters that you're
[01:06:12]
directly going to pass to the function that that produces the command.
[01:06:17]
It almost seems like that could be an opportunity in people's like build or test setup scripts
[01:06:24]
to just, I don't know, have certain modules where you define your real effect handlers.
[01:06:30]
And then you could just derive it from a pretty simple like copy paste of that module and
[01:06:37]
then amend the imports of HTTP to import simulated effect dot HTTP as HTTP.
[01:06:45]
So that could be an interesting thing to explore too.
[01:06:47]
Yes, but we're starting to get to the level of effort where personally, like, you're going
[01:06:52]
to invest doing that work for your application.
[01:06:55]
Yeah.
[01:06:56]
You're getting in touch with me and working out a plan where we can actually get rid of
[01:07:01]
the need for this effect type completely.
[01:07:02]
That'd be so great.
[01:07:03]
I get the feeling that you want some contributions.
[01:07:09]
Yeah, well, yeah, there's just a whole handful of things that I can see the potential for
[01:07:17]
the API from program tests to be even nicer than it is and much easier to understand.
[01:07:24]
So yeah, there's a couple big projects that if folks are interested in, get in touch with
[01:07:28]
me although it would require a bit of commitment to kind of work through some design issues
[01:07:34]
and get this implemented.
[01:07:36]
So getting rid of the need for that effect wrapper is the big one.
[01:07:40]
Adding support for more commands that aren't currently represented as simulated effects,
[01:07:45]
like keyboard focus, the viewport scrolling, things like that.
[01:07:48]
If folks are interested in helping with that, I'd love to hear from them.
[01:07:53]
And it seems like you track these as issues in the GitHub.
[01:07:57]
Is that the place to look for them?
[01:07:59]
Yeah, partly.
[01:08:01]
Certainly if there's something that's missing, either if you are going to work on contributing
[01:08:04]
to it or not open a GitHub issue if there isn't one already.
[01:08:08]
But also not everything is in there.
[01:08:10]
Some of the bigger projects I haven't tracked there yet, not because I don't want them there
[01:08:15]
just because I don't have enough clear information that I want to put down yet.
[01:08:19]
But yeah, that another big one is that the test dot html module that's part of the Elm
[01:08:24]
test package has some missing features and is really composable in a nice way.
[01:08:31]
So there's some improvements there that could allow the simplification of some of the Elm
[01:08:35]
program test stuff.
[01:08:37]
Like currently, if you click a link, you have to both provide the text of the link and the
[01:08:41]
URL that it's supposed to be.
[01:08:43]
But theoretically, you should be able Elm program tests should be able to get the URL
[01:08:48]
from the virtual top.
[01:08:50]
But it's just not possible the way that test dot html is implemented right now.
[01:08:55]
So that's another kind of big project.
[01:08:57]
But if anyone's excited about it, I'd love to chat about it.
[01:09:00]
And small stuff is like you're interested this morning, if you have any improvements
[01:09:05]
to the documentation, or even just issues about some part of the documentation that
[01:09:10]
is unclear, file an issue about it or make a PR with improvements.
[01:09:14]
If anyone's been using Elm program test, or is going through the process of learning it
[01:09:19]
and is inspired to write a blog post, that would be really useful because I've put a
[01:09:24]
lot of work into the documentation, but also I wrote it so I don't really have that perspective
[01:09:29]
of someone coming to try to use it.
[01:09:31]
Some more examples of that would be would be great to have.
[01:09:34]
Awesome.
[01:09:35]
So let's give people some resources to get started here.
[01:09:40]
So you mentioned the docs that you wrote, which are very thorough and well worth a read.
[01:09:46]
Even if another blog post would be helpful, they've got a lot of really good information.
[01:09:51]
So check out the guidebook, which is in the show notes, and just the Elm documentation
[01:09:56]
for the package itself.
[01:09:58]
Also be sure to, you know, there's a lot going on in the HTML test assertion helpers in the
[01:10:07]
Elm test package.
[01:10:10]
So we've got a link to that in the show notes as well.
[01:10:13]
Definitely check out that and familiarize yourself with the API.
[01:10:16]
And any other resources?
[01:10:19]
Is the Elm test Slack channel a good place to ask questions or discuss that?
[01:10:24]
Yeah, I think that actually is a good place.
[01:10:27]
Is it the testing channel?
[01:10:28]
Oh, yeah, just called test.
[01:10:30]
It's called hashtag testing in the Elm Slack.
[01:10:34]
On the docs, I did want to mention two big thanks to some of my former, you know, Red
[01:10:39]
Ink colleagues, specifically Katie Hughes and Michael Hadley and Brooke Angel, who helped
[01:10:45]
review the documentation, give suggestions, helped me improve that when I did the big
[01:10:50]
3.0 release.
[01:10:52]
Great stuff.
[01:10:53]
And Vanessa also gave a really great talk about writing testable Elm.
[01:10:57]
She talked quite a bit about using Elm program tests and some of the accessibility features
[01:11:01]
that you were discussing, too.
[01:11:03]
So that's worth a watch as well.
[01:11:05]
So cool.
[01:11:06]
Well, I mean, there's so much more we could get into, so many great details to talk about
[01:11:11]
here.
[01:11:12]
But this was a lot of fun.
[01:11:13]
Thank you so much for chatting with us, Aaron.
[01:11:14]
Yeah, thanks for having me.
[01:11:16]
And until next time, talk to you later.
[01:11:19]
You're in.
[01:11:20]
See you.
[01:11:25]
You