spotifyovercastrssapple-podcasts

elm-watch with Simon Lydell

Simon Lydell joins us to discuss his new Elm dev server which has a minimalist philosophy and maximal refinement.
September 12, 2022
#65

Transcript

[00:00:00]
Hello, Jeroen.
[00:00:02]
Hello, Dillon.
[00:00:03]
And today we are joined by another repeat guest, friend of the show, Simon Lydell.
[00:00:09]
Simon, welcome back.
[00:00:10]
Great to have you again.
[00:00:11]
Hi, thank you for having me.
[00:00:13]
Our pleasure.
[00:00:14]
Excited to dive in today.
[00:00:16]
So, Simon, what are we talking about?
[00:00:18]
Today we are talking about a new tool that I made, which is called Elmwatch.
[00:00:24]
Elmwatch is a CLI tool that watches your Elm files for changes and recompiles as they change with hot reloading.
[00:00:33]
And the selling point here is that I tried to make it as fast, robust, and delightful as the Elm compiler itself.
[00:00:42]
Ooh, delightful. That sounds good.
[00:00:44]
Yeah, sure does.
[00:00:45]
I love that.
[00:00:46]
Do you want to talk about why you got started with this project?
[00:00:49]
Yeah, so I have the luxury of being able to use Elm at work.
[00:00:56]
And our front end is like 99% Elm.
[00:01:00]
And at work, we were using Parcel for everything front end, including compiling Elm and using Parcel's hot reloading, etc.
[00:01:11]
And at first it felt like it worked really good, but then we started to notice bugs in Parcel's implementation.
[00:01:19]
Like if you started Parcel while you already had Elm compilation errors from before, and then you fixed your errors,
[00:01:28]
and then you went to the browser to see how it worked, you just got a blank page.
[00:01:33]
Because Parcel didn't work if you started with Elm compilation errors.
[00:01:38]
Everyone just learned that, oh, yeah, I need to restart. Okay, here we go.
[00:01:43]
And we started to see more and more of those things like, ah, it feels like it stopped watching our files,
[00:01:50]
or the hot reloading didn't work, so you refreshed the page just in case.
[00:01:56]
And it was also really slow.
[00:01:58]
So after a while, it started to feel like, why do we have this big tool Parcel when our code is 99% Elm?
[00:02:06]
Couldn't we just go with just Elmake? Because Elmake is so fast and has so nice output.
[00:02:14]
But I quickly realized that it kind of sucks to type in your editor, switch to your terminal,
[00:02:22]
and run Elmake again, and then go to your browser and refresh.
[00:02:26]
So I felt like, nah, I really need a watcher.
[00:02:30]
Because sometimes you also forget to rerun Elmake, and you go to the browser and like, what?
[00:02:36]
Why does this link not show up? It's there.
[00:02:40]
So then I started thinking like, well, I could just run Elmake in a watcher tool.
[00:02:47]
Watch all the Elm files, run Elmake when any of them change, and that should be it.
[00:02:53]
Sounds simple.
[00:02:55]
Yeah, it sounds simple.
[00:02:57]
But the biggest thing I missed was hot reloading, which is like, yeah.
[00:03:03]
Can you explain what hot reloading is for people who might not know?
[00:03:06]
Yeah. So for example, if you're working on styling, then it's really nice to put your editor on one side of the screen and the browser on the other.
[00:03:17]
And then you change some Elm code, like a color or how big a spacing should be, and you save.
[00:03:25]
And then it instantly updates in the browser without you having to refresh or do anything.
[00:03:30]
That's hot reloading.
[00:03:32]
Right. So you might have navigated into some application state where there's a modal window showing and there's something you've navigated through to get to some deep part of your application.
[00:03:44]
And then you want to change the UI there, and you don't want to have to click all the way back every single time you tweak something in your style.
[00:03:51]
Yeah, exactly. Modals feels like the usual example.
[00:03:56]
Yeah.
[00:03:58]
So good. But I also started feeling like, I mean, we could switch from Parcel to something else.
[00:04:05]
Also, we were using Parcel 1, and we tried to upgrade to Parcel 2, but it was a lot of work.
[00:04:11]
And it also felt like, okay, there's a chance it will be better, but there's also a chance it won't be because Parcel isn't concerned with Elm in the first place.
[00:04:22]
It's focused on TypeScript and CSS and JavaScript, not Elm. That's more of an afterthought.
[00:04:29]
Were you only using Parcel for the watch mode and hot reload? Were you not using it to bundle the application?
[00:04:37]
Yeah, we used it to bundle like final 1%. We had like a little bit of TypeScript.
[00:04:42]
We used a couple of MPM packages that we talked to via Elm ports and a little bit of CSS.
[00:04:49]
But I started to feel like I don't want Elm inside of that. We can have something on the side.
[00:04:55]
It doesn't matter super much for that 1%. As long as the Elm works really, really good, I'm going to be a lot more happy.
[00:05:01]
Yeah, that trust element of your dev tools is so huge.
[00:05:05]
I mean, we, Jeroen and I talked about that on the building trustworthy tools episode.
[00:05:10]
And that's something we've thought about a lot is if you start doubting your tools, then I'm sure you experienced this with kind of these white pages in Parcel where you start asking, like, am I in a weird state because of Parcel or because of something I did with the code?
[00:05:27]
And of course, with like Elm, if it compiles, it works. You might lean towards mistrusting your tool, your build tool more than Elm.
[00:05:36]
So it feels strange to have this non deterministic weird like that's more the feeling of working with like JS build tools is having having node modules get into weird states and stuff.
[00:05:50]
And with Elm, it just feels so much simpler to just compile and build your dependencies.
[00:05:55]
Exactly. Almost felt ironic. Like we're so proud of the Elm compiler. It's fast. It gives you beautiful output. And what is the first thing we do? We wrap it up in a giant JavaScript framework that is slow and makes it ugly.
[00:06:10]
Right. So this is very interesting. I am I thought about I've thought about a lot of similar things.
[00:06:17]
And we've talked about some of this in some in some chat, Simon, but with the Elm pages dev server. So I'm really interested to kind of compare notes with you.
[00:06:26]
I actually went through like originally I had a similar mentality of kind of bring your own bundler for the JS CSS side in Elm pages.
[00:06:37]
And I think that was sort of like in the web pack days when I was just like, there are so many different opinions that people can have about bundling.
[00:06:47]
And there are all these sort of JS frameworks that give you a way to hook into the low level web pack build configuration.
[00:06:55]
And it feels like a leaky abstraction. It feels like a foot gun waiting to happen because you've got this framework.
[00:07:02]
It's using web pack to do certain things, but then it's exposing web pack so you can do these low level things that might interfere with the framework things that you're doing.
[00:07:11]
And also, what if someone doesn't want to use web pack? What if they have to do some other thing now?
[00:07:16]
They have to fit a square peg into a round hole. And it's just very awkward.
[00:07:20]
And so I didn't and I had that same thought that like we have this beautiful Elm ecosystem.
[00:07:27]
And if you use Elm tools, if you use Elm CSS and you know all the end like minimize your use of JS dependencies, then you don't need a complex build process.
[00:07:40]
So why impose that on every Elm pages user? So the Elm pages dev server and build process has to be this bloated process.
[00:07:48]
And and then VEAT came along and that actually changed my mind on that.
[00:07:54]
And the reason is because for one thing, VEAT is kind of standards based, which I know it actually was inspired by Parcel in that regard, where it says, well, we know how to compile TypeScript.
[00:08:07]
We know how to compile SCSS. Why do we need to have this clunky configuration that everything feels customized and assumes that it's nonstandard and needs to be customizable?
[00:08:20]
Why can't we just say, hey, you have a TypeScript file. I know how to deal with TypeScript.
[00:08:25]
And so VEAT is much more standards based. It's much more performant and it sort of dominated the market.
[00:08:33]
And that made me decide to integrate VEAT directly into the Elm pages dev server.
[00:08:39]
So I'm curious to hear your thought process there, because you're kind of building this tool in the in the days after we've moved away from Webpack and and to VEAT being the dominant player.
[00:08:53]
So what was your thought process there?
[00:08:55]
Yeah. So I actually like the philosophy behind Parcel and VEAT, which is like it's easy to get started with.
[00:09:05]
You can just say like, I guess Parcel and VEAT works kind of the same like Parcel index.html.
[00:09:13]
And it's going to like find all your JavaScript and CSS and even Elm files from there and just take care of it all.
[00:09:22]
But like in my experience, it feels like that's such a huge task that there are going to be bugs and problems.
[00:09:32]
And well, maybe now VEAT feels like the most stable one I've ever tried. I've only tried it quickly.
[00:09:40]
But even then, when I tried VEAT just the other day, there was like one thing I really didn't like about it.
[00:09:47]
And that's when you get an Elm compilation error in the terminal, it prints it in all red.
[00:09:54]
Yeah. Red on black background.
[00:09:58]
You have really messed up.
[00:10:02]
Exactly. And we have this colorful, nice error messages from Elm. It's such a shame that it's just all gone.
[00:10:10]
And I actually talked to someone on Slack. I think his username is Christoph P.
[00:10:17]
He contributed to Parcel too to improve the Elm situation.
[00:10:21]
And I asked him, is it possible to get the colors? And he said, yeah, I tried it, but the way that the error reporting works in Parcel, it just doesn't fit in.
[00:10:32]
And that's why I felt like I want a watcher that is made for Elm first, not something where you try to fit it in afterwards.
[00:10:42]
And you have to live with the tradeoffs of like, yeah, we won't do things just for Elm.
[00:10:50]
Elm will have to change instead of the bundler.
[00:10:54]
You said Elm first. So do you intend on supporting JavaScript or other languages?
[00:11:01]
Oh, no, I should have said Elm only, I guess.
[00:11:05]
Right. Yeah. I mean, there are definitely cool tradeoffs you can make when you say this is a tool for Elm.
[00:11:13]
And again, that was originally what I did with Elm Pages was I said, hey, you can compile TypeScript.
[00:11:20]
And it's just that Elm Pages doesn't know what that is. So you need to give it a JS file.
[00:11:25]
But you can be running a TypeScript watcher in parallel and then point it to the output and Elm Pages will happily take that output.
[00:11:33]
But it only knows JS and CSS.
[00:11:36]
So what I ended up doing with Elm Pages now with the V3 Alpha is I use the VEAT SSR mode, server side rendering tooling, which they've sort of built for framework authors.
[00:11:49]
And what I do is I don't use VEAT at all for compiling Elm.
[00:11:54]
So the Elm code is all compiled completely through my own custom process.
[00:11:58]
I do the whole Elm hot code snippet injection and compile Elm, have the status indicator showing that it's running a hot reload in Elm and all of that without VEAT help.
[00:12:11]
And then I give VEAT this JS entry point and it just thinks it's a JS file.
[00:12:17]
It doesn't know that Elm was involved in that process.
[00:12:19]
So that allows me to have a lot more fine grained control and it also allows me to do things like print colorful error messages and all of that in the overlay, which I run myself.
[00:12:29]
So it's very interesting.
[00:12:32]
But I think the approach you've taken also makes sense and is very reasonable.
[00:12:38]
But VEAT changed my opinion of what makes sense for Elm Pages dev server.
[00:12:44]
And I also think it makes more sense for Elm Pages because it has a much bigger scope or like bigger ambition.
[00:12:53]
I've really tried hard to make Elm watch like as small as possible.
[00:12:58]
It's basically just Elm make in watch mode and I don't want anything else on top of that.
[00:13:05]
That's cool.
[00:13:06]
I like that.
[00:13:07]
I like that philosophy.
[00:13:08]
Did you intend the tool to work for other tools like to be used inside of another tool or do you expect to only be used by people who want to do Elm make on their applications?
[00:13:20]
You mean like if Elm Pages wanted to use?
[00:13:23]
Oh, yeah.
[00:13:24]
Yeah.
[00:13:25]
I've thought about that.
[00:13:26]
And it feels like there might be someone coming and asking like, hey, is it possible to use this as a library?
[00:13:34]
I want to wrap it.
[00:13:35]
And it's going to feel tempting to do it.
[00:13:38]
I think that it won't be good in any way.
[00:13:42]
That's not what I'm looking for.
[00:13:44]
And there's going to be like endless requests for, oh, can you expose this or can I configure how this works?
[00:13:51]
Because in this context, I would really need to be able to change this and that.
[00:13:56]
So I think it's better to keep it as just a CLI tool and if someone else would want to use it, it's better to copy from it or learn from it and do it your own way.
[00:14:09]
Yeah, sounds good.
[00:14:11]
Yeah.
[00:14:12]
Because Elm Review also has a watch mode and well, if it can be more performance, then I'm interested in having that.
[00:14:21]
So I will probably take a look at whatever you did.
[00:14:24]
Maybe let's discuss how optimized it was at some point.
[00:14:29]
Oh, how you optimize it.
[00:14:30]
But I would really like to know like alternatives for this tool because we have had Elm Live in the Elm ecosystem and we have Elm Reactor.
[00:14:40]
Why did you make Elm Watch when those were already there or what does this one bring on top of it?
[00:14:47]
And why should people move away from Elm Live and go to go use Elm Reactor?
[00:14:52]
Elm Watch, sorry.
[00:14:54]
Yeah, so Elm Reactor, I think that's a really great way to try out Elm for the first time because you only need to install one thing, Elm, and you can focus on the Elm code immediately.
[00:15:06]
You don't even need to write HTML or anything or start a server.
[00:15:10]
It just does that for you.
[00:15:11]
But then you're quickly going to grow out of that.
[00:15:14]
For example, when you need your own HTML, because maybe you need to link to CSS or something, it also doesn't have hot reloading, which is big for me.
[00:15:24]
Yeah, for the when you need HTML or CSS, that's one thing that I keep telling newcomers to Elm like, oh, you can use Elm Reactor to get started.
[00:15:35]
But I always feel like I need to tell them like, oh, but if you ever need to use HTML or CSS, then this won't work and you will have to do it manually and you'll be in pain.
[00:15:49]
Yeah, I think it makes sense to say that from the start, like start with Elm Reactor and know that you will grow out of it and come back to me and I'll let you know of the next steps.
[00:15:59]
Yeah, at which point I'm like, I'm not sure what to recommend.
[00:16:04]
And the other thing is like the feedback loop is so crucial to the learning experience.
[00:16:09]
Like to me, feedback and learning are the same thing.
[00:16:13]
And we need it when we're building a sophisticated production application and we need it when we're playing and learning something totally new.
[00:16:20]
To me, it's just they're the exact same thing.
[00:16:23]
So I want tools with great feedback from the very start when I'm learning a new set of tooling.
[00:16:29]
And so I feel like that caveat to Elm Reactor that it doesn't have hot reloading is pretty crucial because you want to be able to get up and running and then not have to constantly hit refresh.
[00:16:42]
You want to see the error messages changing in real time.
[00:16:46]
You want to see the application changing in real time.
[00:16:49]
So maybe like maybe just like a simple maybe a simple starter repo that that beginners could clone could be cool that has like Elm watch with a counter example.
[00:17:00]
That's a standalone repo. It just has Elm watch as a dependency. You npm install. It gives you everything you need.
[00:17:06]
You just you just clone and then run one command. Maybe that would be a nice resource as well.
[00:17:11]
I mean, like you only need if you want to have this experience with Elm watch. The only things you would need is Elm Elm watch and a index study HTML file that is set up for your project.
[00:17:24]
Right. Maybe that's the bare minimum.
[00:17:27]
Yeah. Maybe that's just something that Elm watch could do for you.
[00:17:30]
Like, please create an index dot HTML file that will work. And let's get started with that.
[00:17:36]
Maybe it could do that. I don't know if that's out of scope or if that's if that is a very naive thought.
[00:17:44]
You mean like it would tell you to do that when you do like Elm watch init or that it actually creates a file for you.
[00:17:51]
Yeah. You say Elm watch. Please create index subcommand and then you can get started with that.
[00:17:59]
Yeah. See what you mean.
[00:18:01]
I've been thinking about such things. I've even been tempted so many times to like have like an Elm live mode.
[00:18:10]
We can come back to Elm live for Elm watch where it's like it's super easy to get started.
[00:18:16]
I'll create your HTML. I'm going to start serving your files and so on.
[00:18:22]
But I know that it's just going to grow and grow in like the number of features I or someone else wants.
[00:18:30]
And I know that I'm not going to be able to make something good there.
[00:18:35]
I want to focus on only Elm compilation and not deal with the other things,
[00:18:40]
even though it's really tempting to have that get started easily feeling and just creating an HTML file.
[00:18:48]
It's going to be full of bike shedding. Like which meta tags should we have in there?
[00:18:52]
Stuff like that.
[00:18:54]
Yeah. But speaking about Elm live, so I've used Elm live a lot. It's another CLI tool that watches your Elm code.
[00:19:04]
It has hot reloading. It also has a web server. And I don't remember. Does it do anything with JavaScript and CSS?
[00:19:12]
Maybe hot reloads to CSS. I don't remember.
[00:19:15]
I don't remember either, but it works with CSS and HTML, I think.
[00:19:19]
Yeah. Right. But it definitely doesn't do any transpilation.
[00:19:23]
It's been great to use for like a small demos or toy projects. I've run into many bugs with it,
[00:19:31]
but I think there's a fork called Elm Go, which probably is better.
[00:19:35]
And I also looked into using it at work when I wanted to replace parcel.
[00:19:40]
But the biggest limitation I ran into is that at work, we have not only one Elm map, we have like 10 or something.
[00:19:48]
And I didn't find a good way to run 10 Elm maps at the same time with Elm live in an easy to use way and so on.
[00:19:57]
And these are these 10 Elm maps. They're sort of over the course of a day.
[00:20:02]
You're maintaining you're maybe fixing a bug or adding a feature in two different apps back to back switching between them.
[00:20:10]
It's usually that you focus on one app and then maybe you have one more open because they work together, but they also share code.
[00:20:20]
So whenever you change some shared code, you want to know, did I break anything in any of the others?
[00:20:27]
OK, so that's something I double down on in Elm watch.
[00:20:32]
I really wanted that experience where it's super easy to have many Elm apps and always know if all of them compile, but still be fast.
[00:20:44]
That's cool, because that's a challenge that I keep running into in my CI processes in my dev servers is if there are shared apps, like it's actually surprisingly difficult to just know, like, does everything compile?
[00:21:00]
Like if there if there's a mono repo with multiple apps, it's surprisingly difficult to get the answer to that question.
[00:21:08]
So it sounds like if I change some shared code, Elm watch will give if any of those apps ends up having an error, Elm watch will tell you about it.
[00:21:21]
Exactly. And also, this is another reason for making an Elm specific watcher.
[00:21:28]
Because in Elm, we can choose to only type check code. We don't need to compile it to JavaScript. And I've never seen that in any other watcher.
[00:21:37]
They usually always compile and that is slower, especially if you have 10 apps.
[00:21:43]
So what I do is that when you start Elm watch, I type check all the apps, but only type check them because that's super fast.
[00:21:52]
And then as soon as you go to one of your apps in the browser, because of the hot reloading, then Elm watch is going to know that, aha, you actually use this app.
[00:22:03]
And then it's going to start compiling it for real.
[00:22:07]
That's cool. So basically, you use Elm make dash dash output slash dev slash null for the type checking part is what you're talking about.
[00:22:17]
I'm curious, like, it's noticeably faster and you benchmarked it and it's definitely like makes a big impact.
[00:22:23]
I'm just curious to know, just for my knowledge of Elm optimizations.
[00:22:28]
Yeah, I benchmarked it. It's like many, many times faster.
[00:22:32]
Wow, that is really good to know. Cool.
[00:22:35]
Interesting. Do you know about how much like 10 times faster?
[00:22:39]
Oh, no, I wouldn't want to be quoted on how much it is, but it's like with an up to date Elm stuff folder.
[00:22:49]
I think we can type check all of those 10 apps in under a second, whereas I would guess that many seconds for actually compiling them to JavaScript.
[00:23:00]
So now I'm curious about the hot reloading.
[00:23:05]
How does that work? Like, how do you know? How does a page know? Oh, I should refresh or I should change this code.
[00:23:15]
How does it know that there's an Elm watch process going on in the background?
[00:23:20]
And is this using the same Elm hot as Keith Laszukas or is it a custom one?
[00:23:28]
Yeah, so let's if we start with the first question, which is how does it know when to update?
[00:23:35]
Yeah, it's web sockets. So I after I've compiled your Elm to JavaScript, I put some extra JavaScript at the top of the file.
[00:23:45]
It's just a small Elm watch client which contains code to connect via web socket to the Elm watch CLI.
[00:23:53]
OK, that's smart. And then I can just send updates over the Web socket like, hey, here's some new code for you.
[00:24:01]
And then the client will start to hot reload that. And I looked into using Elm hot, like you mentioned, which is the library that I think Webpack uses it and Parcel and Vite maybe.
[00:24:17]
I think it's inspired by it, but they have like their own copy or something.
[00:24:21]
Yeah, because they need to use ES build for the Vite one. So they had to tweak some things.
[00:24:27]
So they need to use ES build and therefore they need to use ESM JavaScript output. So they need to transform the Elm output, which is not ESM.
[00:24:38]
I see. So the way Elm hot works, if I simplify a lot, it's basically I send over the new JavaScript to the browser and then it can init your app again with the new code.
[00:24:53]
And then it tries to swap them out. So it, for example, it deletes what Elm has rendered before in the page and puts in the new stuff.
[00:25:05]
And it also, to be a hot reloading where you keep the state, it has to like extract the state from the old app and shove it into the new app.
[00:25:13]
And like the downside with that approach where you like swap everything out is that the DOM has state, like scrolling, for example.
[00:25:22]
So when you switch out the entire DOM node, you're going to lose the scroll state. And you don't always do that.
[00:25:30]
You can't maintain the scrolling of the entire page, but if you have like a smaller area on the page where you need to scroll, that's going to be gone.
[00:25:38]
Okay. What about the approach from the other Elm hot?
[00:25:42]
Exactly. And that was something that annoyed me at work because we have a layout where you don't scroll the whole page, you scroll like a part of it.
[00:25:53]
And every time it went to the top, you have to scroll down again and then you can see your change.
[00:25:59]
Which feels like refreshing again.
[00:26:01]
Exactly. So I felt like I'm going to try making my own approach. And then it struck me.
[00:26:09]
I'm impressed.
[00:26:12]
Yeah, that was a lot of digging into weird JavaScript and thinking very hard, taking long walks and thinking very hard again.
[00:26:22]
So if you think about it, all Elm apps at the very edge are exactly the same. The thing that makes Elm apps differ is that you give it different init functions, update function and view function.
[00:26:37]
Yeah. And subscriptions.
[00:26:39]
And subscriptions. Yeah.
[00:26:40]
So the approach I take is that I have a running Elm app, then I get new code. And then I just say to the running Elm app that here's a new update function. Here's a new view function. Use those instead. And also re render once.
[00:26:58]
Whoa.
[00:26:59]
So it's the same Elm app. It doesn't nuke the DOM nodes. It's the same. Keeps the state. It just calls new functions basically.
[00:27:09]
Basically it uses Elm's VDOM diffing, virtual DOM diffing. And if none of the virtual DOM updates need to be applied because the view that's rendered is the same, those DOM nodes aren't changed.
[00:27:22]
If there's part of the DOM that needs to change, then Elm does what it normally does when it renders a new view.
[00:27:27]
Exactly. It's like any other re render.
[00:27:30]
That is extremely clever. That is amazing.
[00:27:33]
I'm going to need to dig into this more because I use Keith Lazuka's ElmHot for the Elm pages dev server, which has worked very well. But this sounds extremely clever.
[00:27:44]
And I am curious if you've been able to squeeze out any performance optimizations in the process or if you're doing any benchmarking or paying attention to performance there.
[00:27:54]
Oh, you mean if like the hot reload itself is faster?
[00:27:58]
Yeah.
[00:27:59]
Yeah, no, I haven't benchmarked that.
[00:28:02]
I'll be curious to take a look at that. But that's very, very clever. That's a very cool approach. And I like that you're paying attention to those subtle problems.
[00:28:12]
I mean, it sounds like from feeling the pain yourself, which is a great way to do it, of maintaining scroll position.
[00:28:19]
And it seems like a more simple mental model of hot reloading.
[00:28:24]
Yeah, I think so too.
[00:28:26]
I also see that like ElmHot has some special case code for, you know, the nav key you get when you...
[00:28:34]
Exactly. Yes. And if you store it in your model in multiple places, it can actually break it because it will go and search for something of type navigation key, which is actually like some type under the hood that's like a function or something, I think.
[00:28:51]
And it will search for that and find it, but not if it's in your model in multiple places in a deeply nested spot.
[00:28:58]
Exactly. And that's just not a thing in my implementation. I don't need to think about that at all.
[00:29:04]
Wow. So this kind of gets back to your Elm first or Elm only philosophy for ElmWatch, which is like,
[00:29:14]
essentially, if we're making a build tool and a dev server that is only focused on solving problems for compiling and hot reloading in Elm application,
[00:29:25]
then what sort of optimizations can we make? How can we simplify the mental model?
[00:29:31]
And it seems like this is like the result of that philosophy to a certain extent. I love that mindset. That's very cool.
[00:29:38]
Yeah, I like it.
[00:29:40]
I feel like there's a few things that I should try out in Elm Review's watch mode now.
[00:29:45]
For instance, when the configuration changes, I basically say, well, drop everything and restart everything, which is going to be slower than necessary in a lot of cases.
[00:29:56]
I think it makes sense for the configuration.
[00:29:59]
Oh, well, maybe not in Elm Review's case, since you change it so often if you actually work on a rule in your project.
[00:30:07]
Yeah, yeah. Or if you're tweaking it or yeah.
[00:30:11]
Yeah, I mean, number one is to be reliable and trustworthy and give the same result no matter how you arrive there.
[00:30:19]
Which is surprisingly hard, right?
[00:30:22]
It's extremely... There's no shortcut to it. It takes a lot of care and a lot of attention.
[00:30:30]
But then the second thing is to optimize it and to do these clever tricks. Like one in the Elm pages dev server, one clever trick I found through lots of benchmarking was that the Elm make process...
[00:30:45]
My code generation was like I compiled two Elm applications, one for doing kind of a headless mode that's resolving data sources and one that's building a browser application.
[00:30:59]
And for the code generation part, I would kind of create a new Elm.json file each time.
[00:31:07]
And it turns out if I write over the Elm.json file, even if it's the exact same Elm.json file content, Elm really doesn't like that.
[00:31:16]
Elm thinks that it needs to redo a whole bunch of work if that timestamp on that file has changed.
[00:31:23]
So I ended up doing an optimization where I will only write changes if it will result in a different Elm.json file.
[00:31:32]
And that shaved seconds off of the Elm make process.
[00:31:37]
It's amazing that you found that. It must be a really intense debugging session.
[00:31:42]
Yeah. Oh, yes. Oh, yes. Lots of long walks, as you know.
[00:31:48]
I create an Elm.json every time, put it in a temporary folder.
[00:31:53]
You may benefit from this optimization.
[00:31:55]
Yeah, maybe.
[00:31:57]
I feel like we're...
[00:32:00]
We are a unique niche in this conversation right now of people who have thought a lot about these random optimizations. Love it.
[00:32:09]
Another thing that I thought of when we're talking about hot reloading is that I said that it's like especially nice for styling.
[00:32:17]
So you're sitting there and tweaking colors, spacing, fonts.
[00:32:21]
But sometimes when you do that, you change from blue to red and nothing happens.
[00:32:26]
And then you wonder like, huh, is it broken or did I change the wrong place?
[00:32:31]
And that's something I really wanted to get away from.
[00:32:35]
So I felt like, hmm, can I do hot reloading better than what most tools do?
[00:32:42]
What can I do to improve that situation?
[00:32:44]
And what I came up with is to...
[00:32:47]
You know how I mentioned how I inject like some WebSocket code into Elm's JavaScript?
[00:32:52]
There I also inject some code that adds some extra little UI to the browser, almost like the Elm debugger.
[00:33:01]
It adds that blue little thing in the bottom right corner.
[00:33:05]
I add a little UI in the bottom left corner, which tells you with an emoji like the status of the compilation.
[00:33:12]
Is it successful or error?
[00:33:15]
And also the time, the last time it was hot reloaded.
[00:33:19]
So if I change something from red to blue, then if I look at the browser immediately, I will see the emoji changed into an hourglass.
[00:33:27]
It's working.
[00:33:28]
And then it changes back to a check mark.
[00:33:31]
It compiled.
[00:33:32]
And you see how the timestamp updates.
[00:33:35]
So you can see, oh, it updated just now, but the color didn't change.
[00:33:39]
Then it must be me changing the wrong place.
[00:33:42]
It's not the fault of the hot reloading or anything.
[00:33:45]
Right, right.
[00:33:46]
Because as we talked about in the building trustworthy tools episode, like people can only trust things if you give them feedback to let them know that you've done something.
[00:33:57]
Otherwise, if even if you do the right thing, if you don't give the feedback that you've done it, people will doubt it.
[00:34:04]
So they need to they need to see that something's changed.
[00:34:07]
Also, another state that you have in that little badge in the bottom left corner is the server disconnected state, which is very important because sometimes like the server falls over for whatever reason.
[00:34:20]
Node processes can get killed or you can stop the process without realizing.
[00:34:27]
And then you have a tab open that had was being served by that dev server.
[00:34:32]
And your WebSocket connections will pick up that it's disconnected and it shows the disconnected status.
[00:34:38]
And then when you restart the dev server, it reconnects.
[00:34:41]
And those those little details make the experience of using it so much better and more trustworthy.
[00:34:47]
It's very, very polished.
[00:34:50]
Yeah, I was going to ask about that. That's very cool.
[00:34:53]
So does it then. So when you restart the server, does it cause a rerender if there was a need to?
[00:35:00]
Well, so it's when you're disconnected, it's going to try to reconnect with exponential back off.
[00:35:08]
So it tries more and more seldomly. And you can even click the little UI and there's like a reconnect now button if you want to.
[00:35:17]
And then finally, when it reconnects, it's going to say that like, hey, I'm an Elmwatch client for Elmwatch version XYZ and I was last compiled at this timestamp.
[00:35:33]
And then this Elmwatch CLI or the server is going to respond with like, all right, you're up to date.
[00:35:41]
Everything is fine. Or it's going to say, aha, you're a bit old.
[00:35:45]
Let's be recompiled for you. And it's going to do that.
[00:35:48]
Very, very nice.
[00:35:49]
There's a wide gap with developer experience between like 95% polish and 99% polish and 100% polish.
[00:35:58]
Like if you have a few rough edges, you can really feel them.
[00:36:03]
And when you go that extra mile and get that last bit of polish, handle that extra set of corner cases,
[00:36:10]
it really changes the experience and your ability to rely on the tool and trust it.
[00:36:16]
Yeah, it really does. And since, I mean, when I started on Elmwatch, there were already alternatives.
[00:36:24]
You could use Parcel, Webhack, VEAT, whatever.
[00:36:27]
So I felt like I need to take my time with this.
[00:36:31]
I have no reason to rush a release because the whole point of Elmwatch is to be stable, fast, not have all these rough edges.
[00:36:40]
So I actually started thinking about Elmwatch in winter 2021 last year.
[00:36:48]
And I thought about it a lot. At first, I didn't code anything.
[00:36:52]
I just had a document called Vision where I wrote down all ideas I had or the vision I had.
[00:36:59]
And then sometime later I realized, oh, these are really bad ideas. Scratch those, change it to this.
[00:37:05]
And then not until in May last year, I started coding.
[00:37:10]
And then it took all the way to March this year until I could release the first beta version.
[00:37:17]
And since then I've released like a dozen of beta versions.
[00:37:22]
And now it feels like really polished and stable where I want it to be.
[00:37:26]
And it means that I could start using it at work and have it be trustworthy immediately.
[00:37:33]
So people don't learn the habit of, I'm just going to refresh just in case or I'm going to restart the watcher just in case.
[00:37:39]
No, you don't need to do that. It should always be correct. Otherwise, tell me and I'll fix it.
[00:37:46]
Yeah, that's one sentence that you don't want to hear from colleagues or users like, oh, I'm just going to refresh it.
[00:37:54]
Like, no, all my hard work.
[00:37:59]
If you release an initial version that doesn't have that trustworthiness, then you might never earn it back.
[00:38:06]
You might never fully be reliable in people's minds.
[00:38:10]
So that's amazing that you've put so much care into the release process, too.
[00:38:16]
It's also something that is like people not having trust in the tool is also something that is very that that is shared among people.
[00:38:24]
It goes from one person to another like, oh, like we use Elmwatch.
[00:38:28]
Oh, by the way, it doesn't always work. So you're going to have to refresh a lot like this contamination.
[00:38:37]
Or have you guys heard of this like social experiment where you have you take somebody in a waiting room and it's like a it's like a fake waiting room.
[00:38:48]
It's like this experiment, experimental setup. But these participants of the test are sitting in the waiting room.
[00:38:55]
They don't have context. They don't know what's happening. But they're actually like a few other actors in the waiting room sitting down next to them.
[00:39:01]
And then there's like a bell that goes off or something.
[00:39:04]
And when the bell goes off, people just stand up and then they sit back down every time the bell goes off and the actors are doing it.
[00:39:11]
But then the actual participants look around and they see people standing up when the bell goes off and they do it because everybody else is doing it.
[00:39:18]
But then you you keep flowing people through the waiting room and eventually the actors go out of the room and there are only actual real people, participants.
[00:39:28]
And they pick up those superstitions of like standing up. They have no idea why they haven't talked to anybody about it.
[00:39:35]
Right. But the same thing applies to tooling that like you can you can really hand down these superstitions and magic incantations you do about tools and mistrust you have for certain things, which may not be founded anymore.
[00:39:48]
It might be passed on from from a time that it was relevant where it no longer is. But we get this way of passing down our superstitions and mistrust.
[00:39:58]
Yeah, I think there's a well, in your story, the they start with only actors and like one participant.
[00:40:07]
There's also something about pigeons where you you feed pigeons during almost randomly.
[00:40:14]
And if you do it multiple times or something, pigeons feel like, oh, well, when I was doing this thing, I got fed.
[00:40:22]
We got we got we got feed. And so you can see them doing the same thing over and over, like flapping their wings or doing circles or something, because they think, oh, well, this is how I will get my food.
[00:40:38]
Some details may have been smudged over there.
[00:40:42]
I mean, I think it's a perfect analogy, because basically, like when tools aren't reliable, then we will do random things.
[00:40:52]
I mean, as programmers, right? Like, I think we all have that intuition about things might not work in a certain state.
[00:40:59]
Like, you know, when when family members ask us for tech support and it's like, oh, probably close the browser window and open it again or restart your computer or what?
[00:41:09]
You know, like you enter a form and like the form has a validation error.
[00:41:14]
And then it's like, oh, well, you autofilled that. And why don't you like manually enter the thing with the validation error, which like you shouldn't have to.
[00:41:23]
But you build up a superstition of these random flapping's of your wing and things that over time you build up these these things that seem to to make the web gods happy, I guess.
[00:41:38]
Speaking about restarting things, so like one of the main goals with Elmwatch is that restarting it should never make a difference.
[00:41:49]
And that's like a really nice goal, but it can also be really hard to do, as we touched on before.
[00:41:56]
And I could tell you one like really interesting quirk in that space.
[00:42:04]
So it has to do with, I mean, a watcher, right? What does it do? It watches files. But which files do you watch?
[00:42:15]
So, I mean, you could start from the complete scratch. I'm going to watch the entire hard drive whenever anything changes recompile.
[00:42:25]
Even all the temporary folders.
[00:42:27]
Yeah, exactly. It's going to be super wasteful.
[00:42:31]
So then you can simplify one level and say, OK, I'm going to watch everything in the project folder.
[00:42:39]
Then it's more reasonable.
[00:42:41]
Yeah, but still, there's going to be annoyances. I mean, you might have a big node modules folder.
[00:42:48]
And if you install with MPM while running, probably don't want to recompile for all of those.
[00:42:54]
So you could say, OK, let's watch all files ending with.elm in the current folder.
[00:42:59]
Well, that's better. But you're probably going to have tests that don't affect your application.
[00:43:06]
And it's like you don't need to recompile when you change those.
[00:43:10]
Or we have the Elm stuff folder. Elm test generates code and puts it in there.
[00:43:17]
But we don't need to recompile our app just for that. That's wasteful as well.
[00:43:23]
And maybe Elm review puts Elm files in there as well. I don't remember.
[00:43:28]
No, it puts it into a temporary folder.
[00:43:30]
Ah, I see. So, OK, we could try to exclude test. We could exclude Elm stuff.
[00:43:37]
But then we have this multiple app situation. Like I said, we have like 10 apps.
[00:43:42]
Then if I change something, let's say we have like a customer app and a admin app or something.
[00:43:50]
If I change something in customer app slash main.elm, then I don't need to recompile the admin.
[00:43:57]
I just need to recompile both if I change some shared code.
[00:44:01]
So now we get into like how do I know which Elm files correspond to which app?
[00:44:08]
So Elm watch has like a small parser that just parses the imports of all your Elm files to know the import graph, everything.
[00:44:19]
And that way I can know like, aha, this Elm file changed. It only affects these apps. So I'm just going to go and recompile those.
[00:44:30]
But that's not as easy as you think, because if you have like an import that says import HTML, that probably means the Elm HTML package, not your code.
[00:44:46]
So like, but the thing is, remember how I wanted, or let's put it this way.
[00:44:52]
If you have import HTML and you create a file called HTML.elm in your code, what's going to happen?
[00:45:00]
Well, Elm is going to say you're importing HTML here, but I don't know what you mean.
[00:45:05]
Do you mean the package or do you mean your file? You need to fix that.
[00:45:09]
And at first I didn't think about this situation. So if you created HTML.elm, Elm watch did nothing.
[00:45:18]
It just thought that, oh, it's an unused file. I'm not going to do anything.
[00:45:21]
But then if you restart Elm watch at that point, now you're going to get an error when you start saying, huh, I don't know which one you mean.
[00:45:30]
What are you up to?
[00:45:31]
So what I learned is that it's not just existing files that affect your apps. It's also files that could be.
[00:45:41]
So for every package import, I also have to know that if you create a file with the same name in your code, then I need to recompile.
[00:45:51]
So you get that error. And that might sound silly. Like how many times are you going to do that?
[00:45:58]
Like why do you spend time on that edge case? But that's like the level of quality I went for with Elm watch. I should like handle everything.
[00:46:06]
Yeah, I can also imagine that you have a source slash environment Elm file in multiple projects and they all get imported by their main file.
[00:46:19]
But like when one changes, which apps depend on this one?
[00:46:24]
So I'm guessing you keep the module name and the source file and somehow related to the Elm JSON file that it relates to.
[00:46:34]
What I actually do is that for each app, I keep a set of file paths that are related to it.
[00:46:44]
Either files that exist or file paths that could exist. And if they come into existence, then I know that, oh, this is going to affect this app.
[00:46:54]
Actually, when you create a file, an Elm file in a project that does not get imported anywhere, do you type check it? Because does Elm make type check it?
[00:47:07]
Excellent question. I don't type check it. You might be able to, but the approach I went with was that at first I didn't think about it.
[00:47:16]
So I just ignored it because like it's unused. But then I realized that I can do better.
[00:47:22]
So the thing is that maybe you create a new module and you work on it for like an hour and you feel really good about it.
[00:47:30]
Your editor says that everything compiles and you're like, oh, nice. Now I'm going to try this out in the browser. And you're like, huh? What? It's like no difference.
[00:47:39]
And then you realize, ah, I never used it. So what I do in Elm Watch is that if an Elm file changes that isn't related to any app, then I actually print a log message saying like this file changed.
[00:47:57]
And for your information, it's not imported anywhere. So I'm not going to do anything.
[00:48:02]
So is it only in the console or also in the browser?
[00:48:06]
That's only in the console.
[00:48:09]
Okay.
[00:48:10]
So you still have to go there and have a look. But I mean, there's a chance you go there.
[00:48:15]
That's very useful already. Yeah.
[00:48:17]
Are there any situations where having an error in an Elm file which is not imported can lead to an Elm compiler message? Because I feel like I've seen it if there's like a syntax error or something in an Elm file.
[00:48:31]
I'm not sure if that's only for like Elm packages or some special case, but I feel like I've seen that before somewhere.
[00:48:40]
I would say for packages.
[00:48:42]
Maybe. Yeah. Which have special rules like there are no source directories for packages. It's just source.
[00:48:48]
But that's really cool hearing the lengths you've gone to do these optimizations and give this feedback and make sure the states are always in sync.
[00:49:02]
And I mean, again, this like Elm only philosophy that you have, like, it might seem like a lot of corner cases, but Elm is a very constrained language.
[00:49:13]
And there are far fewer corner cases than you would have analyzing what something depends on in a JavaScript project.
[00:49:20]
So you actually can do that because you're saying this is Elm only.
[00:49:25]
You actually can handle every corner case and say I can actually build a tool that handles every single corner case.
[00:49:33]
And it's actually possible to build one that is always in sync with what the Elm compiler will say.
[00:49:38]
I'm going to need to learn from some of these things for the Elm pages dev server because I also I have also thought about analyzing the dependency graph looking at the imports of relevant files.
[00:49:48]
And I haven't gone to that length, but I'm going to have to look more deeply at your code because that's very cool. And in the Elm pages dev server, I am.
[00:49:57]
One thing I do is I try to get that above and beyond experience of people trusting the tool.
[00:50:03]
If you have a data source, like let's say if you depend on a markdown file in an Elm pages data source, the Elm pages dev server adds watchers for for the markdown file that you read.
[00:50:15]
So it so the Elm pages data sources is running through the Elm pages engine.
[00:50:20]
So it knows what files it needs to go read to resolve the data sources so it can add watchers to those files.
[00:50:27]
So just by consuming a data source that reads a markdown file, the Elm pages dev server knows to add a file watcher to that specific file.
[00:50:35]
And if you change it, your page will hot reload not just with code changes, but with data that's changed and it will go and repopulate that data.
[00:50:43]
But that can be a little bit expensive because I don't know exactly like in some cases, if you don't touch code related to these data sources, then why go to the trouble of resolving all of that and rerunning the data source?
[00:50:58]
It's a lot of extra work if you've just changed your view to change some styling and the data shouldn't change.
[00:51:03]
So I've thought about this optimization to look at the dependency graph by looking at all the import statements.
[00:51:10]
But it felt it felt too ambitious, but this is making it feel like maybe a little more approachable, maybe giving me enough courage to give that a try myself.
[00:51:19]
So very cool stuff.
[00:51:21]
Yeah, it doesn't sound as scary as one might think first because of Elm syntax.
[00:51:28]
Right.
[00:51:29]
Imports can only be at the top and you can't really do much at the top.
[00:51:33]
It's basically you can just say module name, exposing imports and comments.
[00:51:39]
So that's all your parser needs to know.
[00:51:41]
As soon as you hit something else, you can just bail out and skip the rest of the file.
[00:51:45]
I mean, you can even do it with the reg eggs, right?
[00:51:47]
It's unfortunately impossible because we have in Elm, you can have comments inside comments like block comments.
[00:51:54]
Nice.
[00:51:56]
Simon, you are that is so impressive that you're like ready to go with these like random like niche questions.
[00:52:04]
I love that.
[00:52:05]
I don't think we can stump Simon.
[00:52:07]
He's like thought through all of these details.
[00:52:11]
Yeah, well, that's why I work with ASTs in Elm review as well, because like reg X is like they work, but not when you have comments.
[00:52:21]
And I love that the answer Simon's answer isn't like reg X is just seem like a bad idea.
[00:52:27]
It's like, oh, actually, it's impossible because of this quirk of Elm syntax.
[00:52:30]
That's very cool.
[00:52:31]
I did, you know, another optimization I've thought about with Elm pages is like if the dev server could detect which relevant parts have changed,
[00:52:40]
like if the update function or init function or view function or the data source have changed and do optimizations based on that.
[00:52:48]
I'm not sure if that would be as relevant in Elm watch, but it is really cool.
[00:52:53]
Like how we could use sort of intelligent static analysis to look at the AST and do less work in a dev server because of that.
[00:53:05]
Yeah, and I bet that if Elm watch was like built right into Elm, if it was part of it, then we can have better resolution.
[00:53:13]
So like now Elm watch is file based.
[00:53:16]
If the file changes recompile apps that imported, but it could be function based.
[00:53:23]
Like you change a shared file, one function in it, and that function is only used in one app.
[00:53:29]
Right.
[00:53:30]
Recompile just that one.
[00:53:31]
That would be cool.
[00:53:32]
Exactly.
[00:53:34]
Oh, but then you have to do resolution of module names like, oh, is this function the one from the module that just changed or yeah.
[00:53:44]
Yeah, but I bet that like Elm probably knows that while it type checks and compiles and everything.
[00:53:51]
Yeah, the compiler would.
[00:53:52]
Yeah, but it does need to do a little bit more work to figure this out.
[00:53:57]
Yeah.
[00:53:58]
Also, speaking of that, like that would be my dream.
[00:54:01]
I've sometimes thought about that, like forking the Elm compiler to add the watching stuff in it and change nothing else because then we could actually make the hot reloading even better.
[00:54:12]
So the thing with hot reloading is that like if you change your view function, then sure, you can go ahead and hot reload.
[00:54:19]
But if you change the model or in it or specifically the model, then it might not be compatible anymore.
[00:54:28]
For example, you in your old app, you might have a field that is a number.
[00:54:33]
And in the new app, you have a field that feel is now a string.
[00:54:37]
So if you're going to try to call a method on the number that doesn't exist on the string, I mean, your program is going to crash.
[00:54:46]
But the Elm compiler would know that the type has changed.
[00:54:50]
So it's not safe to do a hot reload.
[00:54:53]
We need to refresh everything instead.
[00:54:55]
Yeah, actually, it's possible to do that in part.
[00:54:59]
So both Elm watch and Elm hot does that you can run in it again.
[00:55:04]
Then you don't get the model type, but you get a model value.
[00:55:08]
And you can see like, has it changed at all?
[00:55:11]
Then we should probably refresh either because the model is not compatible or because like you've changed in it.
[00:55:18]
Then you probably want to see those changes.
[00:55:20]
But the thing is that you can't always know if you have a field that is a maybe, for example, and you always start with nothing.
[00:55:27]
Even if you change from maybe into maybe string, that in it value is still going to be nothing.
[00:55:33]
So you wouldn't know.
[00:55:34]
But I mean, if you knew that the type has changed, then you could actually refresh even in that case.
[00:55:40]
That makes sense.
[00:55:41]
I had an interesting conversation on Twitter once I was asking Dan Abramoff about some changes.
[00:55:48]
He was tweeting about some changes to like the React hot module reloading.
[00:55:53]
And I was asking him some questions because like Elm has really interesting abilities to do hot module reloading because of like its purity and all this stuff.
[00:56:02]
Of course, like Dan Abramoff has thought about these types of things because he built Redux, which was sort of inspired by the Elm architecture.
[00:56:11]
And anyway, he was saying, you know, hot module reloading is all sort of a lie no matter what, because what if you change the types of messages your app can send?
[00:56:22]
Then you may be in a state right now.
[00:56:25]
You're hot reloading that state, but it would no longer be possible to arrive at that state with the messages and the event handlers you have in your application.
[00:56:33]
So it's a philosophical question.
[00:56:35]
But in practice, really, we're not using hot module reloading for that stuff.
[00:56:41]
And honestly, like I would almost say it would be reasonable to just completely give me a fresh, clean slate.
[00:56:51]
If I ever touch my message type or my model type, just throw everything away.
[00:56:57]
That's fine, because the time I actually want hot reloading is if I'm tweaking the view.
[00:57:03]
That's that's really what it's useful for.
[00:57:05]
Now, in Elm, we actually we could do that right.
[00:57:08]
If we're watching sort of the AST or if we're a fork of the Elm compiler that has special access to that information anyway, we can track that.
[00:57:17]
We can say if the message type has changed or the model has changed and just throw it away.
[00:57:22]
So that could be an interesting approach that is kind of not not playing any tricks.
[00:57:28]
And, you know, it's not a hack.
[00:57:31]
It's just if the if the model and messages don't change, then you get your views refreshing.
[00:57:37]
Yeah, but then you would also have to look at has my update function changed?
[00:57:43]
Because that one can stop triggering a message or it can cause a trigger or cause a message with a different value in one of the arguments, which might be a string.
[00:57:54]
You're right. Yeah. So then you would have to keep all the messages in your stack, which would make your page slower, I guess.
[00:58:01]
I think what we currently have is looking very good already.
[00:58:05]
It is in practice. It's fine.
[00:58:07]
And if you really care about like some subtle UI state and you're changing your messages in your update function,
[00:58:13]
chances are you're probably going to like just refresh your page when you start going through a flow or you're going to have some like Cypress tests that check it or something.
[00:58:22]
Right. So I feel like we should maybe talk about how to use Elmwatch because, for instance, there is an Elmwatch in it and an Elmwatch.json file.
[00:58:32]
So maybe can you go through like how to use Elmwatch and what those files are for?
[00:58:38]
Yeah. So Elmwatch can be installed with NPM. And once you got it, you run Elmwatch in it, as you said.
[00:58:47]
And that simply creates a file called Elmwatch.json, which is the configuration file for Elmwatch.
[00:58:54]
And I've tried to make that one as minimal as I can. I don't want like a massive webhack.config.js thing.
[00:59:01]
So all you do in there is specify which apps you have or targets, as I call them, which are the input Elm files and what do you want the output JavaScript file to be called.
[00:59:14]
And it fills in an example for you. So you can just edit that instead of like looking up the JSON structure.
[00:59:20]
And then off you go. You can just run Elmwatch hot and that will start the watcher with the hot reloading.
[00:59:28]
Or you can run Elmwatch make that will build all of your targets, just like Elm make would basically.
[00:59:37]
But not start a server.
[00:59:39]
Exactly. Only build them for production, basically.
[00:59:42]
Okay. So the point of that would be that Elmwatch knows which applications you have and how to build them.
[00:59:50]
So just build them all full one at once for me.
[00:59:53]
Yeah.
[00:59:54]
So you don't have to do Elm make multiple times.
[00:59:56]
Yeah. You don't need to duplicate your config inputs and outputs.
[01:00:02]
And also you can run them all easily after each other.
[01:00:07]
And it's like it even prioritizes them and tries to run it in parallel where it makes sense.
[01:00:16]
I remember that when we had you over for Elm tooling back in that day, you also had a target or a what is my main the files that you need to compile when you want to run this application.
[01:00:31]
And I think he removed that at some point because it was not useful or something.
[01:00:35]
And now you have it again in Elmwatch.
[01:00:37]
Yeah, that's very interesting, actually. So you're talking about the Elm tooling dot JSON file.
[01:00:42]
Yes.
[01:00:43]
So back in the day, we had like ambitious plans for that file that it could be used for shared things across tools.
[01:00:52]
And back then it was used for Elm tooling to install things.
[01:00:58]
And also you could specify entry points. It was called.
[01:01:01]
Yes, but that was only used by the language server.
[01:01:05]
So now that I started making Elmwatch, I was thinking like, hmm, this entry points, it's almost what I need. It specifies the files, but it doesn't specify the output JavaScript files.
[01:01:17]
So I was thinking like, hmm, what should I do?
[01:01:19]
Shall I like extend that?
[01:01:21]
And at first I actually used Elm tooling dot JSON file for Elmwatch.
[01:01:26]
But then I realized like, hmm, the language server doesn't actually need entry points.
[01:01:31]
It can calculate those. So I made a pull request to the language server to get rid of that.
[01:01:37]
And then I deleted that from Elm tooling dot JSON.
[01:01:40]
And I realized that doesn't make sense to have this shared configuration.
[01:01:44]
It's better to have your own file that you own yourself and choose the format, can update as you want.
[01:01:50]
So that's why I went with a new file and not kept going with that.
[01:01:57]
When you say that the language server could compute it itself is because they can look at, find main declarations, right?
[01:02:05]
Why can't Elmwatch do it?
[01:02:07]
Yeah, it depends.
[01:02:09]
Like there's also this case where Elm supports having multiple inputs to the same output.
[01:02:16]
So you can have multiple Elm apps bundled into the same JavaScript file and they will share the Elm runtime.
[01:02:23]
And that it wouldn't be able to figure out.
[01:02:25]
You have to explicitly say, I want these three together, for example.
[01:02:29]
And the language server doesn't care about that because it only needs to type check.
[01:02:33]
Exactly.
[01:02:34]
And those inputs, wouldn't those inputs also in some cases change the type checking?
[01:02:39]
Because you can include in the input something that is not an entry point module, like an ENV.js, right?
[01:02:47]
So if you like, can't you do that?
[01:02:49]
Like if you import an ENV.js, you could point to prod slash ENV.js or you could point to dev slash ENV.elm, prod.env.elm or...
[01:03:01]
No, that would have to be in the source directories.
[01:03:04]
Oh, right.
[01:03:06]
No, but isn't there something where you can bypass that with elm make?
[01:03:09]
I thought there was some sort of edge case where you can actually have an entry point that's not in source directories or something.
[01:03:16]
Maybe I'm wrong.
[01:03:18]
I thought there was some edge case like that.
[01:03:20]
Maybe Simon's going to have to go on a long walk and think about this or maybe I'm just remembering wrong.
[01:03:25]
I remember there being like somehow you can tell elm to compile things outside source directories, which is super unexpected.
[01:03:34]
And I don't think you should do that.
[01:03:36]
I don't think so either.
[01:03:38]
But I think that that's how you can do that is by including it in the inputs.
[01:03:42]
Yeah, but then it's going to type check it, but it's not going to include it in the bundle.
[01:03:46]
It includes it.
[01:03:48]
I think that if you have your source directories contains src and that's it, and then you have main.elm and you import env and your src directory does not contain an env.elm.
[01:04:02]
I believe you can compile your files with elm make src slash main.elm space dev slash env.elm.
[01:04:15]
And now, even though env is not in your source directories, it picks that up and compiles it as if it were in your source directories.
[01:04:24]
That's like a weird edge case where the inputs can actually change an elm file that would compile that would compile with an error.
[01:04:34]
If you didn't include that can now start compiling because you added another input from an outside source directory.
[01:04:41]
Well, no one made an issue about it on Elm Review, so I don't think it's all that used.
[01:04:47]
Maybe now that you've shared that, people will create issues.
[01:04:52]
It's definitely like the dark arts, like things you probably shouldn't do.
[01:04:56]
Like, I don't know, I guess this one's maybe a little less controversial, but I know some people like to do like import list.extra as list.
[01:05:05]
I don't know. To me, that's dark arts and I don't like doing that.
[01:05:11]
Yeah, I do it all the time. Oh, no.
[01:05:16]
I don't like it for multiple reasons, but that's another topic for today.
[01:05:22]
We'll have to have a spicy opinions.
[01:05:24]
I think I actually talked about it on some episode, but yeah, I wouldn't know which one.
[01:05:30]
But circling back to June's question, how to use it. So I said how you could run Elmwatch init and then Elmwatch halt or make.
[01:05:42]
But the thing to keep in mind here is that Elmwatch is basically just a thin layer on Elm make.
[01:05:50]
So that's the mindset you need to have.
[01:05:52]
It will create a JavaScript file like Elm make would, and there stops the responsibility area of Elmwatch.
[01:06:01]
It's your responsibility to create an HTML file, load that JavaScript file, open it in your browser or serve it in a server and so on.
[01:06:13]
And I mean, if you really don't like that or if you prefer a tool that does everything for you, even if it means somewhat worse Elm experience, then you can choose that trade off.
[01:06:27]
But you shouldn't come to Elmwatch and expect it to be like parcel. I'm going to do everything for you.
[01:06:33]
It's more like it's going to be like Elm make.
[01:06:35]
And just to clarify, it's not that it gives you one Elm.js output file, but it gives you one per target.
[01:06:43]
So if you have 10 target applications, you give it the output just like the dash dash output flag in Elm make.
[01:06:50]
You give it the file name to output to and it will output that for each of your targets you specify the output.
[01:06:57]
Yeah, that's how it works.
[01:06:59]
So there's one interesting feature that you have because you say that it's very simple, but you also have a thing called post process.
[01:07:09]
Yes. Do you want to explain what that is?
[01:07:12]
Yeah. So if you've ever used Elm like at work or in a more serious setting than just a demo or side project or something, you might run into some issue with Elm that is really important to you.
[01:07:31]
Like a famous one is that if you use browser.application or browser.document, then Elm is going to render inside the body tag.
[01:07:40]
But if you use Elm in like in at work, you're probably going to have maybe a chat widget that is a third party script that also put something in body or your customers are going to use a browser extension that puts something in body.
[01:07:55]
And that can cause Elm's virtual DOM to like not understand what to do and crash.
[01:08:02]
And there are some known workarounds for this, but they usually means that you need to patch the JavaScript output from Elm.
[01:08:09]
And the annoying thing with using, for example, webpack or parcel is that then you need to figure out how to do that.
[01:08:15]
And it usually means writing a webpack plugin and learn their API.
[01:08:20]
And it's going to be really annoying. Or you could try to do it like after webpack when it's written, it's JavaScript file, try to do like do regex on it or something.
[01:08:29]
That has two downsides than working with a minified code. So all variable names are just going to be like A, B and C instead of like virtual DOM underscore render underscore download or something.
[01:08:42]
And also you won't get those patches during development. And that's not good because then like you've modified the code only for production and you don't run that so much locally.
[01:08:53]
So maybe it doesn't work. So I wanted to make it really easy to be able to, for example, patch JavaScript, the JavaScript output all the time.
[01:09:04]
So I have this concept of post processing Elm's output, which you can use then for patching.
[01:09:12]
You can also use it to minify the code for production. And like that doesn't really hit the things that Elm watch wants to do. I mean, I want to be simple.
[01:09:24]
But since I had this post processing anyway, it's really nice to put it in there because Elm watch will then run everything in parallel when you make when you when you compile for production.
[01:09:38]
If you minify yourself, you're probably going to write like a bash script that does everything in sequence. And then it takes much, much longer than if you minify five things in parallel.
[01:09:49]
So would the post process script be run before or after your patches?
[01:09:54]
Oh, no, I'm actually blanking on that.
[01:09:57]
Yes, I got it.
[01:10:00]
It has to be after my patches, actually, because you can minify the code and then I won't be able to patch it. I won't have any clue what anything is anymore.
[01:10:11]
So that's a caveat. You will be working with slightly modified code by Elm watch.
[01:10:18]
So the post process will you can only have one, right? You only have one script that you can run after.
[01:10:26]
Yeah, the post process thing and these will always run.
[01:10:31]
So when if you say like you say this script will minify the thing, it will even do it in debug modes, right?
[01:10:39]
Because you can do Elm watch dash dash debug or run it without the optimized flag, which you can also add.
[01:10:46]
Yes and no. It's true that the post process script that you use always runs, but you also get to know which target you're working with and which Elm mode you're working with, like debug, standard or optimize.
[01:11:05]
So what you would typically do is check like, am I in optimize mode?
[01:11:11]
OK, then do let's do minification. Otherwise, only do these patches.
[01:11:17]
OK, the post process script will get the dash dash debug or dash dash optimize flag as well.
[01:11:23]
Exactly. Yeah. OK, because I was the first time I saw this, I was like, oh, is this for enabling Elm optimize level two?
[01:11:32]
Yeah, because as you say, like this will change the code in production mode and I would like to test this out, but I don't want this to be run in debug mode.
[01:11:42]
So, yeah, exactly. You want your post process script to be really, really fast when you run in standard or debug mode, because otherwise your hot reloading is going to be slow and that's super annoying.
[01:11:55]
But if you do want to try the production stuff like with minification, with optimize level two, if you want to, then you remember that browser UI that I talked about that I inject in the lower left corner.
[01:12:08]
If you click that, you can actually choose which mode you want debug, standard, optimize.
[01:12:14]
A run time? Yeah. OK, interesting. What happens is that I send a message back to the Elm watch web socket server and says, like, hey, I would like to be recompiled in debug mode and it's going to do that and then send it back over.
[01:12:29]
And that gives you two cool things. One is that it's like really easy to turn on the debugger.
[01:12:36]
And the other thing is that you can turn on optimize mode, which will actually run the minifier so you can like run that if you would like to try it out.
[01:12:46]
Amazing. I guess you will then refresh the page just to start in a clean slate, but and it will still be enabled.
[01:12:53]
Oh, that is very cool. Like, especially being able to have the debug window.
[01:12:59]
That's a big thing to optimize. It's super slick. It's really cool how it like instantly refreshes when you click one.
[01:13:06]
It just recompiles it and loads everything. And yeah, also, like I love this.
[01:13:12]
It's very, very clever, this this sort of post process, how you get the compilation mode. And so you can actually use that as you know what you sort of switch on to decide how to post process things for production or def mode.
[01:13:29]
And then you can test out your optimized output and it's actually running the same process.
[01:13:34]
And you're actually running ElmWatch to do your production builds. I think that's that's huge. And that's actually like I've learned a lot from the philosophy behind Vite.js.
[01:13:45]
That's like one of the insights that I know Evan Yu was like really big on was he was like, I don't want like a Webpack dev server that behaves totally differently.
[01:13:55]
And it's its own bespoke thing that isn't related to the production build. That's just madness. Like I will I want things to go through the same sort of setup.
[01:14:07]
And maybe there are some production, some additional configuration for production.
[01:14:12]
But I want it to be using the same thing. So I don't like so I have some confidence if I'm running it in dev mode that it will work in production.
[01:14:19]
And so, yeah, I think it's super slick that you've kind of built it that way.
[01:14:24]
And this also ties in with like how Elm doesn't fit into other build tools sometimes.
[01:14:33]
Because all build tools that I've used is they only have two modes, development and production.
[01:14:40]
But Elm is a bit unique in it having three modes, you know, standard mode, debug mode and optimize mode.
[01:14:47]
And I mean, in a perfect world, there would be only two modes, debug and optimize.
[01:14:52]
But in the real world, unfortunately, the debugger can like slow down your page.
[01:14:57]
So you can't always have it on. And that's the word we have to adapt to.
[01:15:01]
So having a watcher built for Elm, I have the possibility to be able to switch between three modes instead of like having a hacky environment variable that you need to flip to turn on the debugger and stuff like that.
[01:15:15]
Right. Yeah, when Elm is a first class citizen, it changes the sorts of optimizations you can make and building the tool around that.
[01:15:23]
And then when other things are second class citizens, when you say JS is a second class citizen, there's even more you can do to improve that Elm centric experience.
[01:15:33]
I want to make sure we touch a little bit also on like what what what does it look like if somebody does have TypeScript, SCSS, Tailwind, some sort of external processes they need for their other CSS and JS assets?
[01:15:50]
You've got a really, really nice example in the Elm watch repo, which we'll share a link to. And you talk about using another tool you built called run Pty.
[01:16:03]
You want to talk about what's the best practice for that?
[01:16:06]
Yeah, I can talk about that. So that example is actually very similar to what we use at work. And it's Elm watch plus ES build. ES build is a super fast, easy tool for compiling JavaScript, TypeScript, CSS.
[01:16:25]
But the thing is that like one nice thing with like web hack and parcel is that you can just turn on one tool and get going.
[01:16:34]
I don't know if you remember like back in the days, I used to like have one terminal tab where I ran CoffeeScript watch, and another with like SAS watch.
[01:16:43]
And it was annoying to switch projects because like you had to close all those terminal tabs and open all of them and the next one.
[01:16:51]
And that's why I built this little tool called run Pty, which lets you start several watchers in one terminal tab, and you can close them just with one keystroke.
[01:17:04]
But it still it, it still lets you look at the output from one watcher at a time. It doesn't mix the outputs. And that's really important to me.
[01:17:14]
Because I want to have a bit of Elm output and a bit of TypeScript and a bit of CSS. I want to like have them cleanly separated.
[01:17:22]
So you use interactive commands to switch between them?
[01:17:25]
Yeah, exactly. It's run Pty is an interactive terminal application and you can switch between the different commands you're running and choose which one you want to see and interact with.
[01:17:36]
Interesting. So I could use it for Elm test watch and Elm review watch for instance.
[01:17:41]
And that's made me realize like, I used to think that I liked parcel because like, oh, nice, I have one watcher for everything.
[01:17:50]
But that's not it. That's not what I liked. I liked that I could have one command to start everything.
[01:17:56]
And having one watcher for everything is actually a downside to me because then you get this mixed output.
[01:18:02]
I want to have TypeScript error in one place, Elm error in one place and so on.
[01:18:08]
So to me, this run Pty workflow works much better.
[01:18:13]
And that goes to your philosophy, which you talk about in the Elm watch read me that you say you chose not to have this sort of, you know, HTML like in browser overlay for errors because you prefer having terminal output because these tools like Elm were designed to give you terminal output.
[01:18:32]
So just show nice, colorful terminal output.
[01:18:35]
So basically your recommended workflow is to have a terminal open with the Elm watch server and have a browser open simultaneously so you can see errors that pop up.
[01:18:47]
Yeah, that's how I like it at least.
[01:18:50]
And that's also because many tools put like an error overlay on top of the page. And what usually happens to me is that I want it to go away. I want to look at what's behind, because maybe I'm deep in a refactor and I don't remember like, hmm, what did happen when I clicked this button after going to that page?
[01:19:11]
And I want to be able to do that. But if the error overlay is in the way, it's super annoying. Even if you can close it, it's like knowing having to close it.
[01:19:19]
It makes a lot of sense. I think the only thing I would say about that is I think that the Elm watch dev server could make it more prominent to like make you really feel that something is wrong.
[01:19:32]
Because right now there's like a little error emoji in the status bar, but like you could have like red text or red box or like a red box around the app or something to like really make it feel like something is off so you don't kind of gloss over that detail and miss the output in the terminal.
[01:19:51]
Yeah, I've thought about that too. It might be a little bit too subtle right now.
[01:19:56]
Maybe just making the whole status box bright red would cue you in enough for that.
[01:20:03]
Yeah, for sure. Elm watch only works for Elm applications, right? It doesn't work for packages. That was my initial... I don't have many applications on my computer.
[01:20:15]
So when I wanted to try it out, I was like, okay, let's try it on one of my packages. And that didn't work. And you explained it well in the remake. Do you want to explain it again?
[01:20:24]
Yeah, that's correct. I've been thinking of packages. Like if you're used to having a watcher always displaying the latest compiler errors, then maybe you want that when you develop a package as well.
[01:20:39]
But I realized that I don't do that many packages. So when I tried to shoehorn that in, it didn't feel good. And I also realized that you can sort of rely on your editor if you have Elm type errors in your editor.
[01:20:57]
And it's also for packages, you can just type elmake with no arguments and it will type check everything quickly. So I felt like, nah, I shouldn't support packages. At least not for now. It doesn't really make sense.
[01:21:12]
And I don't know... I don't have this use case myself, so I wouldn't be able to make something as good.
[01:21:18]
Yeah, for me, it's mostly that Elm test watch mode will do pretty much the same thing anyway. And that works really well. So even though I'm like, ah, Elm watch should support this, I'm like, yeah, well, Elm test does it.
[01:21:35]
I think that's a good choice for the scope. The one thing that I can't help but think about is for this sort of bring your own bundler approach for the JS and CSS. Like it seems like the one thing you miss out on, the run PTY sort of solution for running things in parallel seems like a really good approach, except that you don't get hot reloads for CSS and JS unless I'm mistaken.
[01:22:01]
Well, that depends on what you pair Elm watch with. If you pair it with Vite, for example, you will get it. If you pair it with esbuild, you can do it, but you have to do it yourself.
[01:22:12]
So at work, we just didn't bother with TypeScript because changing TypeScript almost always means we need to refresh the page because it runs at init.