Large Elm Codebases with Ju Liu

Ju Liu joins us to share some tips and techniques from working with a large Elm codebase at NoRedInk.
February 14, 2022


Hello, Jeroen.
Hello, Dillon.
And today we've got another guest with us.
We've got Ju Liu joining us.
He works at No Red Ink, and he's
going to be helping us talk about
large codebases and maintaining
large codebases in Elm. Ju, thanks so much
for coming on. Thank you for having me.
It's a pleasure. So, yeah,
I'm really excited to have
you on and excited to hear about
your experiences working on
the No Red Ink codebase.
So let's dive in.
Maybe we could get a little bit of background.
For anyone who doesn't know, do you
want to tell us a little bit about what
No Red Ink is, what Elm
looks like big picture, how much Elm
there is at No Red Ink. Let's kind of get a little background.
Sure, of course.
I think for all intents and purposes,
we're an educational platform.
So most of our
users are either teachers
or students. And we've been
writing Elm for our front end
for many years.
I would guess since
2015, but maybe
some other experiments are even
before that. So, and
I would say 99.9%
of our front end
code is written in Elm.
We still have few lingering pieces
of various JavaScript
frameworks just lingering
in the darkness.
But no new code is written
in those frameworks any
longer. Those pieces are waiting to be
To fight the newcomers.
I think, you know,
every time we do
a Hack Day, there's someone doing
okay, I should, you know, remove
that old page which is still written
in React and Elmify
it. So it's so common that we
use the word Elmify. You know, I think
I remember a couple of years ago,
there was a large effort to change
like 10 of these pages and
the whole project was called, you know,
Elmifying, yada, yada, yada.
We're all
trying to Elmify some small part
of the world. That's our...
Elmify people mostly.
That could be
dangerous. Be careful with that.
so architecturally,
no red ink is not a single
page app, a single, single
page app. It's multiple.
It's embedded as elements
on each page, right? Yeah,
that's true. So I think
I should have said that as a
word of warning. Like everything
that I'm going to say today
is going to be very
strongly influenced by that
decision. And I'm sure that
there are applications out there which
are structured as one
whole single page app for
the whole product. And definitely
your experience might
be completely different.
In our case, I think even
at the beginning, we
decided to go through this approach
of creating an individual
endpoint for each page
so that we could more
easily experiment with
Elm at the beginning and
further later with different
approaches on how to write an Elm app.
Of course, nowadays,
many of these apps, they share some
code or some
components or some
sort of way of writing
Elm. But I think this structure
helps us a lot to
work at scale, right?
Right, right. So you
bypass some of the, I mean
it's a different type of scaling Elm
application than scaling a single
page app. You don't have the entire
routing layer and all those pieces are handled
by Rails or I believe
Haskell in some instances.
We use Haskell just as part
of microservices which don't
really serve Elm
apps. Well, I'm saying that
they serve very rarely used
Elm apps.
But the majority
of the routing is done
in Rails. That is not to
say that we do also have single page applications,
but they're just one of the many
Elm apps that we serve.
So we have some
individual pages where we thought that having
an SPA gives a much better
user experience. So those are built
using the SPA
The one by Ryan Haskell, guys?
No, no, no. So I think
when we started writing those,
the one that Ryan built
didn't exist yet.
Yeah, it's pretty recent, right?
So we have our own little
wrapper around
the one provided by
the Elm
Right, I saw that in
your blog post. So we'll link to your
element no red ink blog post.
That's one of the details that I love
when no red ink people
share little tidbits from the
code base. I always
see Richard dropping little
helpers that seem really cool and you
have some in this post where you
wrap things and custom
tailor in a little way where instead
of browser.element you
kind of take in a flags decoder
and these little things that
just make it a little more ergonomic.
Yeah, I think that
makes particular sense
in a case where
there are going to be many people
doing that operation
many times. I feel
like if I'm working on my
own project, I wouldn't take the time to do that
because how
many L maps are you going to create?
One. Yeah, probably
one or maybe two or three.
But if you're in a large app
and especially with regards
to new engineers,
right, like you want to create a page
but you don't really know where to do
what to do. So having a script
that, for example, generates
all this boilerplate for you
it makes things so much easier.
And I think also having like some
little bits of structure
on top of the normal
program that Elm provides
you makes a lot of things easier
such as
tracking events or
being able to pass some more context
into our
applications for many
reasons. One example
I can think of, which I'm sure I mentioned
in the article as well, is
with regards to accessibility because
we want to detect
which is the main way
that the user is using the site
and to do so, we
have to track that in one way or the other
and having that in our program
makes it a solved
problem. You can just create a new endpoint
and you will have that information
with you.
I've been thinking about this
lately. I think that
create like frameworkizing
things, which is sort of what we're talking about
here, like creating little mini
custom tailored frameworks
and for your own purposes
to me it's almost like
a refactoring process
where you see these patterns
emerging where you're doing something
over and over and you just try to refactor
it. Has that
been your experience as you've seen these abstractions
emerge? Do you see duplication
emerging in all of these pages and then
you start to abstract it away?
I think that's definitely
one part of
the problem. I wouldn't
say that extracting duplication
has been our main concern.
I think our goal
was just to try to
make the big problems
easy or make the
hard problems easy. There are
some things which are usually a bit awkward
to set up. I think that
has been our main goal with creating these
frameworks is to do that.
I can still think of some little duplications
here and there that
we haven't really solved and
it's sort of okay
still. Even
with these little wrappers, I can
remember a few projects as
where we still experiment with new
ways of setting up
the program. For example, one of
the recent efforts was to
find a way to handle
models throughout the
whole frontend. Because right
now every individual program
has to handle their own
models, decide what to do when
they come in front or
when you dismiss them, where should the focus
return and things like that.
I think we're just trying to solve
these difficult problems and
figure out if there's an easy way so that
everybody can benefit from those
It's more about finding the
ways that things could go wrong and
trying to create a path
where you don't have to think about those
We developed new things to help
remove boilerplate or ensure
consistency or make some hard things
easier and use that in
one of the pages or one
part of the project.
Since you have a large codebase,
what do you do to make that
use everywhere?
How do you spread that?
Hint, hint, nudge, nudge.
I wasn't even thinking of an answer.
I was just
plainly asking.
I see where you're going.
This is whether I'm going to disappoint
you. We don't use Unreview for
We do have some scripts
where we write down deprecations.
I think this is especially
important for our
UI component
slash storybook
which is called an Arriding UI.
Which is published?
It's published.
You can look it up.
From the repository,
we should also be able to reach
a Netlify page
where you can see the preview of all
those UI components.
I think that's quite a nice example
because all those components are
version. Usually when we
release a new version of
a model, we want to have a way
to keep track where in the app we're
using all the older versions.
We have this little Python script
that we run as part of our
CI suite that keeps
track of that and just writes
it down a file. I think
we used to have some rules like
this file can't get bigger than this or bigger
than that. But I think just having that
information somewhere is quite important
and to be able to
track where
are all these deprecated
usages of
these libraries. Something that I don't
think has been published yet,
but I think Aaron
has been working on it
on and off is a way to do
some of these upgrades in an automatic
I'm not sure how much is available
on GitHub, but
I think one day we will see it
in its true vision and it's going
to be fantastic.
Looking forward to it.
When I asked the question,
I was also wondering,
I wasn't even thinking about Elm Review.
Although it does sound
something that Elm Review can help
with, definitely, even with the
Python script that you mentioned.
But I was thinking, how do you organize
it on a company level,
team based level?
Do you just go
hey everyone,
can you please all start
using this component this way or
start using this tool?
Or how do you do it?
I'm not sure if this
is a company wide
thing, but it's definitely something that
I truly believe in.
If you want to get a
change across a code base,
it has to be automated.
Especially in the
latest years, I've come to this
firm belief that if you want
to have something enforced,
you have to make it part of an
automated process.
At least partially, I'm guessing.
At least partially, what can be
done? But I think if it
can't be automated, then it should be okay
that it's not enforced.
I think I come
from a Ruby on Rails
sort of background. I think there's a lot in the
Ruby community, this idea of writing
beautiful code,
or doing things the right way.
But I don't think there's much focus
into finding a way to
make it
automated. I think in many
of these choices, we've
tried to come up with ways
to define these constraints
in code and make them part
of CI so that
everybody learns
through these
failing tests.
If someone really hates the
idea behind the rule, they
can make a case for it, disable the
rule. That's fine. But I think at least
it makes the conversation very
much practical.
You can look exactly what the code is doing
if there's something that you disagree with.
But I think you lose all this
white shedding about,
oh, but I think this is the right way to structure
In my opinion, you have to accept
the differences then at that point
when you can't enforce it in any way.
Absolutely. Can't agree more.
How do you try to
automate these changes? What tools
do you have? Do you use scripts to
change your code? And how do you do that?
Do you use Elm syntax? Do you use
Elm language
tree sitter? What do you do?
If I think about the scripts we've used
over time, it's just a mishmash
of every possible thing that you can
think of, like simple batch scripts.
Programs that run in Python
that grab
through the Elm files to figure out
import trees dependencies
and shove them into
an SQLite database
to create these
recursive queries where you can figure
out the whole dependency trees.
I've personally written some Ruby scripts
because I couldn't find
something and I just did it.
So I think it doesn't really matter.
I'm sure that
Brian has done a lot of work
with tree sitter. So I'm sure that
now probably the same script is
using tree sitter nowadays.
But I think it's sort
of like an implementation detail
as long
as the script is there
even if it's a bit buggy, it's
fine. As long as the general
idea of that
constraints are enforced.
Are these more flagging inconsistencies
making things consistent
or a mixture of both?
How to say a mixture
of both? I think maybe now
it's not like
I don't want to give this impression
that we have thousands of scripts, but I feel
right now we might be running three
or four and one of them is Elm review.
So maybe we will
talk throughout the episode
maybe later a bit more
in detail about how we
adopted Elm review in the code base.
But I think
some of the things that we're going to talk about
they're also not enforced.
But I think that's
the point I was trying to make that
since they're not enforced through
the code, maybe
we have this tacit
and shared understanding
that this is the right way to do it,
but we're also okay with
other ways of structuring the code.
I don't think you'd ever
encounter in a code review
saying, oh, don't structure the code in this
I feel like that's what we've
sort of agreed upon that
it's basically fine
since that we have the separated Elm
apps also to write code in a
slightly different way. And also it's
just like the same nature of a very large
application is that
every once in a while you will bump into
an Elm app that nobody has touched
for a year.
And everything that now
we think is the right way to
write Elm apps wasn't
at the time. So
I think it's just like a common
to be able to accept
diversity in the code base
to a level where
all the types are passing,
all the tests are passing.
I think there's some
things that we cannot let go
off. But apart from that,
I think we are pretty
accepting of different
ways of writing Elm.
Yeah, the most important thing is it
And to a certain extent,
without diversity and
experimentation, you don't grow and
learn new techniques. So
I could imagine that's part of the
secret sauce of stumbling upon
these techniques that are really effective
over time. And I think also
you end up going back and forth
on beliefs
and ideas. Like every once
in a while you think, okay, I need
to write a library and I need a way to
pass options to this library.
And I really like the Elm HTML
design where you pass
this list of attributes.
But every once in a while
you bump into the other issue that,
oh, but what should I do
when someone passes the same option twice
or they pass them
duplicated? And then you
think, okay, I'm going to pass
a map there.
It feels to me like every once
in a while you think, okay, this is the best
solution. And then you think about another
slightly different application and
your best solution is a bit tricky
and you go back to the other one.
So I feel even in
some of our UI components, you can
still see this. Like if you look at the
shape of the API, some of them
they really love this Elm HTML
pattern. Some of them
really like this option of creating
a sort of option
map. And then you have this final call
which instantiates the thing
or renders the checkbox
where you do
this pipeline sort of syntax.
I like the builder pattern.
Yeah, the builder pattern. Sorry, I couldn't remember
the name.
Yeah, I mean in API design
Jeroen and I have talked about this a lot, but you
have to have all those tools
at your disposal because it's really about
different solutions to a problem. And the
only way to really do that is to try a
lot of things and really
deeply think about the problem and
the needs of your particular
domain can change over time.
There's personal preference
in there when there are different ways to do things
to achieve the same solutions
and get the same constraints.
So it's really interesting
how that stuff evolves
within the context of a company's
code base. So what are some
of the things that
you think have been really important
for helping keep a large
code base maintainable?
What have been some of those techniques that you think
have helped you? Maybe the
easiest answer is the Elm
compiler. I feel like that's the
major helper, right?
Especially when you're making a change
to a data structure
or to a component that's very
widely used, or you need
to reshape it in some slight
different way. Maybe
when you wrote that component
initially, it didn't support
usage, right? And now it needs to support
that. So the surrounding app
needs to react
to all these different things
that the component now can do.
Having the Elm compiler
just gives you this
confidence that you can sleep
at night, right?
You make the change, and it just applies
to all the different Elm endpoints
that we have. And
eventually there will be a time
later in the day where
the tests are passing
and the types are compiling, and
then that's done. I'd say
that's the major thing. And
especially if
I think about our choice to write
more Haskell, I think that was definitely
by the difference in
developer experience when
refactoring Elm and
refactoring Ruby. Just like this
idea that you can make this change
across 40 files,
and the change is sound,
and the change is solid. So I think
in that perspective, maybe there's
not a true difference between a large code
base and a small code base in Elm.
At least that's why I was
thinking, I think
yesterday I was mulling
over some of the questions about
what does it mean to work in a large code
base? And truly
now I can't really think
what's the difference really. There
are some differences, but
at the end of the day, the real
thing is that a compiler
is going to go through all your code
and make sure that all these
little constraints that you've sprinkled across
the years in the code base,
all of them, they're all
I think it's also this power of
this compound
interest. That
little assertion, that little
constraint you wrote seven years ago
has been
enforcing that thing since
then. And all the new ones that
you've written today are going to
have this compound effect in five years.
So it's like this difference
between when you invest
1% of your pension
when you're 15 years old,
sorry, when you're 18 years old,
as opposed to investing
20% of your
income when you're 50
years old. The power of
compound interest is just
mind boggling. I love that idea
that you don't see a clear line
between what it's like managing
a large or a small Elm code base.
Because I do think that that speaks
to what's really
fascinating and unique
about Elm is that it
is, it's designed in a way
where you have to deal with all these
cases explicitly. You have to model
these constraints explicitly. You have to,
it doesn't let you get away with
not handling something
or not being explicit about something or
not being precise about something. But then
the benefit is you're working with
a giant code base and it doesn't feel any different
than if you're working with a small code base.
Which is one of the reasons why it's difficult
to kind of share
the joy of Elm with people because
you show people a to do list application
and they're like, okay, well, that's not
that special. But you're like, no, but it's actually
like, go make a change in a large
code base and see
how the compiler walks you through your changes.
You still get that same experience.
Yeah, I think that's like
one of the things, I'm not sure if Richard
has spoken
about it publicly,
but it's like one of those things that
you can only experience when you try
And it doesn't really, you know, like you can
hear it or you can read about it,
but up to the moment where
you try it and you see it, you
just can't believe it. So I think
that's what makes it difficult. Like it's difficult
to say, well, that same feeling
that you get here, you're going to have it
in a large code base. And I think
you know, even before I joined the company, I had
the same curiosity.
Like, well, but how does it feel to
write Elm at a
large scale? And people
used to say, well, we just do it.
I think maybe that's the answer.
Like when you're working on
a daily basis, it doesn't really feel
different from anything else.
You just run Elm test.
Like now in our code base probably
has to run 5000 tests.
It takes a minute. But when that's
done, you know that everything
is connected. All those little
types are connected in the right
way. Not the right data are
flowing through those little pipes.
And once that's
done, you just take it and push it
and you know, go on
with your day. Right.
I think one of the nice things is that
there's a lot of work that you don't have to do.
So the compiler
makes sure that you do all the necessary
work, but it doesn't ask you to do
more. So
when I was doing JavaScript in a pretty
large code base, I can
imagine that I was doing some
refactoring and then the
value that I computed somewhere was being
passed to a very
old, very complex module.
And I would have to look
at the implementation
to make sure that my changes
did not break whatever it was doing
inside there. But with the compiler
I don't have to do that.
I just know if the compiler is
still happy, I'm happy
and we can move on. And I don't have to
look at that old and complex module.
So that is very refreshing.
I think this is extremely important
also because of other
reasons that might not be evident
at first. For example
in our case, we work in distributed
teams. So you
might make a change which introduces
14 compiler errors
and it's
at the end of your day and
you don't want to be working late because
you have to go out
and eat a pizza or something.
I think it's just such a good way
to leave work for someone else saying
okay, I've been going through in the middle of this change
but here are all the places
where the build is broken.
If you have time, can you take a look and
fix all the compiler errors?
And I feel it's just
such a good way also to collaborate
with other people.
I think that's what I want
to stress in this very practical
way. It's not like you have to
write a very long message to try
to explain
the purpose of your
grand change. I think that
happens a lot in other languages
where you have to have
really to communicate your big
plan before someone else can
take on your work.
I feel that in Elm,
if you made the
problematic change that
is breaking the code, then fixing
that becomes a sort of
different task.
It's almost mechanical.
Of course in some places
you have to look at the code
a little bit, see how the
pieces of the puzzle fit.
But I feel that it becomes almost something
that's already done.
It's just
already done in
theory. In practice, it's not done.
But the big part of the
plan has already been done.
The scary part is done. Knowing what to change
and knowing what to watch
out for, that's the
compiler's job. A word that's coming up for
me is contracts because often
teams will have to negotiate the
contract of an API
or like a
public facing REST API
or whatever or
how different interfaces fit
together. When you have
different teams, they have to figure
out those contracts between these pieces
that interconnect.
Elm is so explicit about contracts.
Whether it's communicating
with a server or
interfacing with a
common shared piece of code,
the contracts are so explicit.
When negotiating those things between
teams, it makes it
a lot easier because you have this explicit
definition of these contracts
everywhere you go. Maybe we can
talk a little bit about the testing story
at No Red Ink.
You mentioned I think 5,000
tests and
in this blog post you have about
Elm at No Red Ink,
you talk about some of these details.
Maybe you could walk us through a little
bit what it looks like if you sit down to
implement a new feature.
What does the testing look like for that?
That's a great question.
I love to talk about testing.
I think in general, a lot of people
at No Red Ink
have done Ruby.
I think in the Ruby world,
there's this almost obsessive
love for testing.
The most common
Ruby library is probably RSVAC
or assertions
and stuff.
It's just like the bread and butter of
a Rubyist.
I think that's the reason
why in NRI
we do a lot of testing
even in a place where I think
a lot of people when they think of you have
static types, therefore you don't need
I think we just love tests so much that we just
cannot not write them.
We always start from
what we call an acceptance test.
This is usually written in Ruby.
It uses this library
which drives a real browser.
Right now we're using Firefox.
It spawns a Firefox instance,
opens a copy of the Rails
application, and tries to
click, submit forms,
do everything, run assertions on the page.
I think usually that's almost
at the level of the user
starting. But of course these tests
are really slow because they need to spawn
an actual browser instance.
We try to keep those tests
to only the happy path.
Basically we don't handle errors,
we don't handle possible problems.
We just do everything exactly
that will lead to a good outcome.
We will assert that the page says
your assignment has been created,
and then our work is done.
This is the general flow. Sometimes we will
also write some regression tests
in this fashion if there's some
really critical user experience
Especially I think the moment
where you figure out a very nasty
bug in all of these pages which is
causing a lot of user pain,
we like to write a lot of tests
just to cover that to make sure that it never happens again.
I think this is the
highest level of testing
that I can think of.
Actually that's not true. We even have
Cypress tests which we run
against a live staging
instance of the application.
Is there a reason that you use
I'm guessing like Capybara or something on the Ruby side and then
you have to do a lot of testing
on the Ruby side?
Yeah, I think that's a good question.
I think that we use something on the Ruby side
and then Cypress
for the other acceptance tests.
Is there a reason that you have the split
or if you were doing it fresh today
would you maybe choose Cypress for
all of them?
Yeah, that's a good question. I feel that
I really love Cypress
as a tool, but I feel that it's even
more flaky than Capybara
it also depends on
the fact that that piece of
structure is always there.
I think there is some value in
having Capybara specs
because for example we can
very easily parallelize
Capybara specs. We can create
30 databases and
split the test suite across
30 groups and run them
concurrently. But we can't create
30 exact copies
of the staging instance.
I mean we could, but it would be
more difficult and time consuming
to do so. I still see
there are some benefits in having
ephemeral database
that you just run tests on and
you can control how many browser instances
you want to spawn.
Right, it's more connected to the actual
Rails app so you can do these
fine grained controls as you run these.
And also stub some things, right?
In some cases you don't really want to
make all those calls to that certain
API because you pay
for them or you're rate limited.
So having
the ability there to say do everything
for real except for this thing
is quite useful.
Do you stub or actually perform
queries to GraphQL
APIs in the process? Because that's
one pain point a lot of people run into is
how to do end to end
tests against a GraphQL API.
I think in our tests we just run them
so they just hit the end point and
the end point just comes back with the result.
So we don't do...
I think in that perspective we try to stub
as little as possible. In some cases
this cannot be avoided
but in this level of
testing we try to keep it really
to the least amount
of interference that you
Yep, as realistic as possible
as you go up the
testing pyramid. I noticed in your
post about
[00:32:38] that you mentioned
view tests and I'm curious to
hear your perspective on that because
personally I haven't
found a use case where I've
personally found value with view tests
but it seems like you do
find value so I'd be really curious to hear
what you think is valuable about that.
Let me just quickly backtrack
on that. So I think we talked about
this very high level testing.
Yes, yes.
Down this pyramid I would say immediately
below that I would
have what we call the Elm
program test. I think that's
the right way to call them nowadays
which is the library
that Aaron has been
working on where you can basically
set up the Elm application.
You can assert
that certain commands,
well certain effects have been
produced. You can
simulate the response of the
API by providing some
sort of response and I feel like that
in my mind is the
level immediately
below the Rails test.
This is a test written in Elm.
It's much more reliable.
It's much faster but you have
to sort of connect the real app.
Usually in our
applications we pass some JSON
so that the page initializes with some
data. That's the right moment
for us to also search
that for example
that JSON that we pass
can be decoded
by the Elm application.
For example, I'm not sure if I talk
about this in the blog post but we have
a little bit of code that from
our Rails controllers automatically
generates JSON blocks
and we pass them to these Elm
program tests and they try to
decode it as a first thing.
Just to ensure the sanity that
if we make a big change to a controller
which breaks Elm pages
we notice it very quickly
because I think that's also something else
I'm sorry I'm talking too fast
but Ruby tests are
slow so there's no way you're going
to be able to run them
on your machine at every change
that you make especially
if you're working on your own branch
but Elm tests are fast
which means that if I'm just going
for a glass of water
or going for a little walk I just run
the whole Elm suite
it will finish in
50 seconds so by the time
I'm back from the restroom I can see
if I've broken anything
across the system.
So this is the second level
the Elm program test
and I would say below that
I would see the view test
where I think
the reason why we like those
is when you have very
large views
with a lot of conditionals
and inside the Elm program
test I feel it's a bit awkward
to be able to simulate all the possible
cases of these conditionals
you have to click on this button
then click on the other one then
close this model then do that
and instead in the view test you can just pass
a model and you can even
fuzz the model, right? You can
create a property
that generates thousands of
these models but you want to make sure that
no matter how the model is created
inside the view we can see
this and we can see that.
I see. That's a great description
thanks for that.
So like if you had like
an avatar for a user
and if it's a guest user or
a logged in user or an admin
you would have it displayed differently
you could test that and say
if the user is logged in
we see their name somewhere
and get confidence on
just that piece and have a more focused
test where you don't have to run through
all of the update loops to arrive
there. Yeah and I think
even below the view test
we would have what we normally call a unit
test, right? And I think we reserve
those for the really
algorithmic bits
you know like we have this pretty
large and convoluted algorithm
that does a lot of stuff
and operates with these four data
structures and does some
really interesting bits
I think that's where you write
unit tests because it's mostly
just purely, well
you could argue that even view tests are
pure tests, right? They don't
even actually write them
but I think that's the thing
the more you go down into this
pyramid the
faster tests get, better
you can describe all the
possible edge cases
but you're always like exercising
the test on a smaller
surface. Now the further down
you go, you're just
a pin, the edge
of a pin at some point
while when you think
all the way to the top, the Cypress test
you're actually even testing
the network connection
to the instance that sits
somewhere in AWS.
Yeah, that makes a lot of sense.
So for Elm program tests
has it been challenging to make everything
use the effect type in the context of
No Red Ink or was that
pretty natural? It was pretty
natural. I feel like in some pages
we start using effects before
we start using Elm program tests.
Okay. Yeah. Because of testing
still or for other reasons?
I think maybe for a cultural
reason because in order to
write these wrappers
to the Elm programs
the thing that we discussed at the beginning of the episode
in order to do that
to write a program that wraps
the Elm program, you have
already to do that. You have to create
your own effect because
there are some of these side effects
that you want to propagate
down to the actual app
and then there are some events that you want
to keep at the level of your wrapper.
So for example, let's say
you want to keep track if the last
action done by the user was
a mouse click or a keyboard
press. That event is not
going to be propagated to the user
application. It's just going to
remain at the level of the wrapper.
And if you want to do that, you
have to be able to separate
what is a generic
effect that you want to handle
inside your wrapper or what is a user
effect that you want to propagate down to
the user code. So I feel like that's
just like something that the moment
you get a little bit curious and you try to
say, okay, I want to write something like that.
How does it work? And you try to write one
and maybe it takes you a couple of hours.
But from the moment you do that,
you realize that
it doesn't cost you much.
You just need to have a little function
and you can call it perform.
It goes from an effect
to a command message
and that's pretty much
it. And you can write
your own batching
logic or sequencing logic
of these effects. You can have complete
control over what
to do. I think sometimes on
the Elm Slack, I read people saying,
oh, I have many of these
commands and I want them to behave
in this certain way, but I don't know how to
do that. And I think
the moment you do that, you separate
the side effect, that command
message from your own effect.
You realize you basically have complete control
over what you want to do with that.
Fascinating. Yeah, that's a great
take on that.
Yeah, it really does
give you a lot more
control as a, I mean,
insofar as you're
able to be a sort of framework
author and you can sit,
I mean, I often think about this sort
of like, I think we can build our own
frameworks in Elm and I think people sometimes
don't realize the power
that we have available if
we create our own sort of wrappers around
things like these effect
types that we can get
fine grained control. You can
wrap browser.element
or browser.application. There are all
these things you can do
if you have a wrapper layer, you can even
have sort of a wrapped
state that you can control
as a sort of layer of abstraction
around the page state,
for example, which I believe
you don't explicitly say, but you
hint at in the post where you talk about
having this user session and passing
that in. This is sort of this wrapped
state. So you build your
own framework for your custom
needs. Okay, so
another big question that comes
up a lot in the context of larger
Elm applications would be
managing nested
state. This is like one of the really
big questions you touch on this
in your blog post. So
I guess there are two main pieces to
this. There's sort of web components
and then there's sort of
nested Elm applications.
What's kind of your
take on that? How
do you approach that at Neuridink?
I think this is a really great question
because when I wrote the blog post originally,
I feel a lot of people
around the company, they have different ideas
on what to do. I think
that most of the feedback I've
heard internally about
the blog post was exactly about that
section. So I think a lot of
people with their suggestion
would be don't nest state
at all. Try to create a component that
doesn't need the state in the first place.
And I think Evan
at the time, he wrote this
sortable table component
that I'm sure is still available
that you can check out on how to
a view which is complex
and allows for customization
without the need of storing state
anywhere. I think that's like
one quite brilliant
piece of design. I think personally
I'm more in the camp
of nest
applications. It's fine
if you feel like there's a necessity
to create a new type of message.
You can map
HTML. You can
handle the new message.
Do something with it. I feel like
if you do it once,
you pretty much understand
what the complexity is.
I feel like in a lot
of places we say, oh,
this is too heavy handed
and it requires like a lot
of glue code, but I don't feel
it's that true. Maybe I've just
taken the
red pill so I just can't see
how awkward
it is. But personally,
I'm in the camp of nest
applications where you think
it makes sense.
Definitely the first time you do it, you have
to probably use HTML
map, which is something you
might have never used before.
You're maybe going to spend
a few minutes to figure out what does it
mean to write HTML map?
Because when I'm writing a list map,
I know exactly what I'm doing.
But when I'm writing HTML map,
it sounds weird.
And what is that lowercase msg?
But to be fair, there are some
challenges as well. Like this, how
do you convert the top level message?
The nested
command message needs to be
handled just
in the same way. And I think that's
also something else that
shows the importance of
having effects. That if you
need to have some shared effects
or effects that need to
introspect, it's so nice
when you have your own type because
the command message type is quite opaque.
You can't do anything with it
anymore. Like once you receive it,
the only thing you can do is to propagate
it. So having a way to
inspect that, I think both in tests
and in real code
is quite valuable. Because sometimes
you just don't want to do it straight away.
You want to wait a little bit, right?
Or you want to take a look, peek
inside the effect before
deciding to execute it.
That's actually, you know,
I often say wrap early,
unwrap late. And that actually lines
up with that because if you,
in a sense, you're sort of
creating this type that is
this lowest common denominator.
When you create a command, it's the
lowest common denominator. And you want
to sort of create this type that
you need for the lowest common
denominator to pass to the outside world, whether
it's a JSON type, a command
type. And you want to create
that low level lowest
common denominator type that the outside world
needs as late as possible.
Because like you said,
it limits your ability to
inspect it, interact with it,
modify it if you turn it into the
lowest common denominator right away. And it's
also less constrained because it just could
be many things instead of
having a clear type.
When you said that, I was thinking about JSON
values, right? Like, eventually
there would be a moment where we take our model
and we need to encode it into a
JSON value, which is this opaque
type. But imagine if that's
the first thing that we did, right?
We wouldn't even be able to change the model
after we encode it to a JSON, right?
When it gets to this
base level
of information, then you
can't do anything with it anymore.
You can decode it again.
It's a great pattern.
Every function. Every function
called, decode, encode.
You mentioned this sort of
quill web component in the blog
post and that's come up in other
conference talks about No Red Ink.
How widespread are web components
in the No Red Ink codebase?
Could you maybe give a short summary
of what you use web components for?
Yes, of course.
Basically, web components
are part of the HTML
There are ways
where you can create
basically new HTML tags.
You can create your
own elm dash editor.
You have complete
control when this node
is added to the DOM, how
it needs to react in case
events happen.
You basically have this little API where you
can create your own new tag.
I think in the
No Red Ink codebase,
we use it whenever we want to
interface ourselves with something
that it's awkward to do
in pure elm. I think, for example,
the case of quill is a good
one because we all know that
content editable
is a very difficult HTML
API to work with.
The behavior of the API
is weird. It has a lot of
edge cases. It doesn't behave
in a nice way. I think many people
think, okay, I'm going to create
this little wrapper around
content editable. Then, three
days later, they come back and say,
oh no, this is just too difficult.
I mean, there was the
Implementing Elm podcast, which had a whole
season about this topic.
Right, right.
That's what I'm saying.
It's just very
difficult to do it
in a good way.
Even for a normal desktop browser.
Then you start thinking about
mobile browsers, Safari
mobile, and you just
realize that it's just too much of a
task for a single human
being in its lifetime.
I think this is one of the cases where
using a web component
allows you to use
some JavaScript code that someone else
has written, which handles all of these
API edge cases.
But at the same time, it gives you
this little seam,
this little bridge, and you can
send data to the thing
from Elm, receive data
from the web component,
and be able to integrate
it seamlessly
in your Elm application.
I think a word of warning is that
since it's JavaScript code,
any error in the web component
will cause crashes.
I think that's the
word of warning.
Every moment you get a little bit outside
of the Elm world
and you just start executing
JavaScript code, you open
yourself to random crashes.
In fact, they do happen in some
I can see it in our bug tracker
every once in a while, the library
just hits some edge case and just crashes.
But I think it's quite
really nice. If you've never
done it, I definitely
recommend trying creating a simple
web component that does something.
another case where we use it quite a lot
is, for example, for read aloud.
We want kids to be
able to listen
to portions of the page being
read aloud.
And we haven't spent
the time to write a pure Elm
voice synthesizer.
And instead, we know that the browser
has some good support, so
we just include this web component
everywhere and we pass the data in
from Elm. The web component just
renders itself and
it knows that whenever
someone presses the button, it needs to
read out the contents
of the section of the page.
And we don't have to worry about that.
And it also means that
you don't need to set up
the port interaction.
Right. Yeah.
You can do it declaratively.
And this
also means that, for example, as we
were mentioning before, it just makes some
things hassle free.
You don't need to set up all the
glue logic, sense things
back and forth. You just create it.
You know that this is self contained. We will
do one thing and do it well. Hopefully
it won't crash.
But I think, yeah, it's
really nice way of making
hard things simple.
And even in Elm,
there's some things which are very hard to do.
And if you figure out the right
way to solve this problem, why not?
I feel definitely every
time I write, I work in that
area of the code, I'm three
times more careful
than what I'm writing in Elm code.
It's like every time you're
sort of, you know that
if you make a mistake here, it's going to be
bad. And instead when I'm
making the changes in the Elm code
base, I'm like, yeah, that's fine.
You know, like, eventually some
something will catch the problem. But whenever
I'm writing a web
component, I know that I could be
causing the whole page to crash if I'm not
careful here. You use it in
some key places, but you try to use it sparingly
for a handful of
of these helpers.
And I think especially in these cases,
let's say an editor, it's
really awkward if you want to implement
using a part, right? Because there's so
many events that can go towards
the editor. Now you can write
text, you can select text, you can change
the style of the text, you want to add
an image. There's like so much stuff
that you need to thread
through, right? If you wanted to write a part.
But instead, if
all you're doing is to send some data,
receive some data back that you need
to serialize, then
that's not too bad.
Right. And we often say
in the Elm community how
don't write
components. Components are
an object oriented sort of
paradigm and
write helpers and
pass state around. But
you can write web components.
And that's what web components are there for.
They can encapsulate state. They are
components and you can use them
declaratively and not worry about the state.
That's someone else's problem.
Just like using
you know, if you use
like a detail element from
the browser API, then
you've got this expanded
collapse state. You don't need to
own that somewhere. The browser owns it
and that's good. It's declarative state.
It's actually super fun, right?
Because I think we think of web
components as the part where the JavaScript
exists. But actually there's
this library which allows you to add
Elm bits
to a React application and it does
so through web components.
Inside the Elm lives
inside the web component and
it lives within the larger
context of a React application.
So, you know.
That's awesome.
I mean,
I think that there's room for some nice
tooling around using web components
in Elm in general.
But it would be really interesting
to have some tooling for
creating a little Elm
component and embedding that
in your app as a web component. I wonder,
I'm not sure if that would be a good idea, bad idea, but
seems like...
I've done it at least a few times.
I once had to rewrite an Elm application
to React entirely
and I did it bit by bit. So,
at the start I was injecting
React through web components
in an Elm application and
then more and more. And then at some point I had
to reverse everything. So now it was
an Elm app that was
using React
web components inside it and it was itself
wrapped in a
giant React app.
That was kind of fun.
No, it actually was fun because
I kind of like that. But
the quality did drop because
it was an Elm. But I think that's a
testament to the power of
these HTML API. I think a lot
of times, you know, we're quite critical,
you know, like drag and drop
or content editable.
But I think there are some
HTML APIs there
which are really nice, really powerful.
They're at the right level of abstraction
and they just work
really well. So we should probably link to
the Elm community
slash JS integration examples
repo, which has some examples
with working with web components and
other JavaScript
interop techniques.
I can also link that React
components that I was mentioning.
I'm sure it's like
I can find it. I will search it and
send it to you. That'd be great.
So are there any other
things that
you think would be interesting for
listeners to hear about, Ju, that
you've learned from your experience working
on this large Elm code base?
The thing which I left, you know, as
the last cherry on top.
I think it's Elm Review. I think
we just love it.
In large code
base, a lot of code
just goes unused.
And I'm just talking
about a single Elm Review
plugin package.
But I think that one
for me is the nicest
one. Just to show you how
much code gets
written and just abandoned.
And then for ages it's
lived there in that file and nobody
cares about it. And then suddenly you
have a tool that says, well, all the code
that you don't use, how about we just chuck it?
And then you rerun
the build, nothing fails.
And I think it's just like so
good when you have a large code base
where a lot of the changes that
you make are quite
And every once in a while, someone
leaves the company and all that context is
gone. And we make a little
change. And it's like
all is good when the compiler
compiles. But I think there's
still some loss of
information or there's some extra
information which is not needed
anymore. But when you're trying to regain
the context by reading the source,
you have to still
understand it. And it's so nice to know
that, no, everything that I'm reading
in this file matters.
There's no more guesswork to be done.
And I think that's
what really makes me excited about
Elm Review. I think maybe
it's interesting to
know that you don't have
to apply it
on your whole code base.
So I think that originally
was my problem because I ran it initially
maybe a year ago
on the repository
and it yielded
errors. Some of them
could be auto fixed.
Some of them could not be auto fixed.
So it's a difficult thing.
If you had to
say, okay, I'm going to spend the next
three weeks fixing Elm Review
So I feel like it's really important
when you're in that situation to come up with
a plan. So I think this is
what I mentioned before.
I wrote this very simple Ruby script
that just selected one
little slice of our application
and just applied Elm Review on that.
The first thing I did was to
add that to our suite,
to our test suite
so that the moment
that little slice of the code
is covered, it's going to remain covered.
And someone is going to get annoyed
because their build is going to fail
because they deleted
some code and didn't clean it up.
But I think having this
guarantee just helps you that
you don't feel you're
alone fighting
this big war.
You have these little
things that you clean up and they remain
clean. And I think over the course
of maybe, as I said, like six
months, every once in a while I would
work a little bit on them. But
it's like this, as I said, it's like another
case of this compound interest.
You make this effort, but it's not
wasted. And I can think a lot
of work which I've done in the past
where I try to clean up this pit of
code, and then maybe I can't
really communicate well the intent
I change
my mind throughout and I forget
to go and change the other
code which I've written before.
And having a tool that can do
that, that you can express what you believe
is the right thing to do,
and having it run at every round of the
CI is just
invaluable. Sorry,
I ranted a lot, but I'm really excited
about this. You can
rant for hours.
I'm happy to hear it.
I think
that you would have an easier time
today doing this because some
rules have more fixes than a year
ago or two years ago. Two years
ago. Yeah, it's barely
two years old.
Also, there's the
suppression system now.
I've read in
the change log, it looks cool.
Yeah, I think it would make it a lot
You definitely want to
make sure that everything goes out
at some point. But I guess
this is also related to what we were saying
before, that just because a tool exists,
I feel like I really recommend
people to try to play around with things
and think, this tool is
doing this, but it's not exactly what I like.
What is a very
easy experiment that can run
and I can just
lie to the tool. I can tell the tool,
this is the whole project, just run on this.
this very quick and simple example
will tell you, is this really
what I want? Is this what I need? Before
you commit to making the big change
or adopting
the technology. I think just like this
idea of being able to create
short and small experiment
to validate your assumptions
is very important.
Are there any other
Elm review rules you can think of that
you've been making use of that
have been particularly helpful? Or has
it mostly been the unused stuff
that's been the most valuable?
The unused stuff for me
is the most valuable, like
unused constructors, unused functions,
unused imports.
I've seen some new rules
that are coming out, which I think
they look really good.
I think that's also something that I've
been planning to experiment with.
Even though I want to be
careful also with the runtime
of running Elm review
on the whole code base.
Right now, I think we've gotten to a point
where I just run Elm review.
Maybe it takes, in our
code base, probably takes a couple of minutes
to run. I'm sorry.
that's also why I haven't been
adding all the rules, because I still
want people to be able to have that sort
of medium
feedback loop when you can still
run something on your computer.
To give you a sense of perspective,
our Ruby suite
takes 20 minutes to run
on CI, where we break
it into 20 chunks.
So if you had
to run this locally on your computer
with just a single browser, it would take
400 minutes to run
this thing. So I think that's the
sort of meter of judgement.
I want something that, in the course
of a tea break,
it can give you an answer,
an interesting answer.
And I think up to when it's
at that point, a feedback loop is
still okay. But
anything slower than that, I feel, is
just too much friction
to ask someone to use this
as part of your daily work, though.
Yeah, I definitely agree.
Running Elm review in
my small packages
is just so fun.
But running it on my
work project feels painful
because we have a lot of rules and we also have a
pretty large code base.
And yeah, it takes maybe a minute
or maybe slightly less than a minute.
And it already feels so painful.
partially because I'm the author and I know
I need
to find a way to make it faster.
And I have some ideas, but
yeah, everything takes time.
But I remember that when I
presented Elm review
at the Paris
meetup the first time,
one of the first questions I got was
how fast does it run?
Like, what can you do with it or how
how sound are
the reported errors? It's like, how
fast is it? And I was like, I don't
know. When's the Rust version
coming out?
Is there like a tree seed
implementation of Rust? I'm sure
there is. There must be. I think
there is, but it might
be the tree seeder for Rust
that I'm confusing it with.
But yeah, I think if I think
of the recent changes we've
introduced, I'm
really euphoric
about adding
Elm review.
And it's not just me. I think a lot of
people are saying, oh, this is
really great. Just having
I think it's just this sort of
other mind that takes a look
at the whole code base and gives you this
this other representation of it
saying, okay, these other constraints
which are not enforced by the compiler
that are enforced by this other
sort of compiler. If you squint
hard enough, it's just another compiler
really. And it's
doing some other work.
You can enforce some
other rules.
I think before we were mentioning deprecation,
right? So I think this is
something that Elm review can
do really well. You have this module
and it's calling this method and you
would like to deprecate it in a
soft way maybe at first.
You can very easily write
an Elm review rule.
I think the beauty about this sort of work
is that it's quite infectious.
So from taking inspiration
from Elm review,
we also written our own
rules for RuboCop, which is
the sort of Ruby
linking tool that we use.
And I think if I had never
seen how to write an Elm review rule,
I would have never bothered
writing a RuboCop rule.
But once I've done one,
I'm like, yeah, it's basically the same thing.
You walk through this thing and this
is how the
function call is expressed
in the Ruby AST.
And once you've done
one, looking at the other is just
a game
of find the differences.
Yeah. I'll be curious
in a year to check
back and hear because
I mean, it sounds like the sort of automation
you do running Python
scripts and Ruby scripts and
TreeSitter scripts and that
sort of thing to analyze the
code base and automate things.
I can imagine there will be more
custom no red ink Elm review
rules coming. So I'll
be curious to check back in on that.
And by the time Jaren will
have written completely in Rust
and will run like in one second,
then all
our problems will be gone.
No pressure.
I think
that's the main takeaway of this episode.
Jaren's rewriting Elm review in Rust.
Can't wait.
All right.
To be fair, I feel that
the Elm test runner, which is written
in Rust, is not that
significantly, significantly
faster than the one written in Node.
So maybe there are some problems
which can't be just magically
fixed by Rust.
Yeah, the issue is that
it would take a lot of time to
rewrite it in Rust just
to make sure that there are
More likely Jaren is just going to
make Elm 50
times faster with his
Elm optimized level two improvements
he's been working on and then
problem solved. Yeah, exactly.
All right. Well,
Ju, thanks so much for coming
I believe the last I heard,
No Red Ink has open
positions for Elm
developers and for Haskell developers.
Is that correct? Yeah, that's
absolutely correct. So feel free
to check us out. Yeah, I've been
working here for some time. I'm
still here. So I think that's
That's all we can ask for as developers, like for
a place where you just done, you know,
dread it. And I think
that's pretty much good.
You can work on Elm and it feels
just like working on a small Elm application.
And where can
people go to follow you and find
out more about you? You can just go to my
website. It's called
and that's all. There are all my
little useless blog posts
about useless projects.
And maybe there's a couple of useless
articles, useful articles, but
that's the ratio of, you know,
signal to noise.
Delightful, delightful articles.
Actually, Jeroen and I used your
Elm performance articles quite a bit for
our performance episodes. So lots of good stuff.
Check it out. Well, thanks again for coming
on. Ju, it was a pleasure having you.
Thank you so much. And Jeroen,
until next time.
Until next time.