Intro to Opaque Types

Intro to Opaque Types

Opaque Types are a fancy way of saying "custom type with a private constructor." We talk about the basics, how to get started, and some patterns for using Opaque Types.

PublishedApril 3, 2020

Episode#2

Opaque Types

Some patterns

Runtime validations - conditionally return type, wrapped in Result or Maybe
Guarantee constraints through the exposed API of the module (like PositiveInteger or AuthToken examples)

Package-Opaque Modules

Example - the Element type in elm-ui.
Definition of the Element type alias

elm-ui's elm.json file does not expose the internal module where the real Element type is defined.

Example from elm-graphql codebase - CamelCaseName opaque type

Transcript

How are you doing today?

I'm doing great.

I'm ready to dive into this topic.

What are we talking about?

We're talking about opaque types today.

Since we're going to talk about types a lot during this podcast, I think we need to set

up a good baseline with talking about opaque types.

That's a great point.

It's a big topic and I don't think we're going to be able to cover everything about opaque

I think this is really just setting it up for many future episodes to explore this topic

This is going to be opaque types episode one out of some amount of episodes.

I think this is like a whole...

There's going to be a whole college curriculum where this is opaque types 101, but hopefully

we can get a 201 and a 301.

Or more specified episodes.

Opaque types about phantom types, opaque types about validation, for instance.

Opaque types is really just one of the core tools in your design toolkit as an API designer

When I say API designer, I don't just mean designing packages.

Of course you're using your API design toolkit more than any other time when you're designing

an Elm package.

But really if you're just building something at your workplace or something in a small

project that you're maintaining, you still use API design to make your code easier to

You use API design a lot when writing an application.

So how about a definition?

What is an opaque type?

What I would say is an opaque type is a custom type or a union type in other languages where

the constructors are not exposed to other modules.

That's exactly how I think of it.

It's just a really fancy sounding term for saying that it is a private constructor.

The private constructor for a custom type.

It's not that mouthful but less mysterious.

So what would be the most basic example of an opaque type?

I've got a go to example that I use.

So one thing I use is there are certain constraints that you can't do simply using an Elm type

You need a type that is private and therefore you can ensure certain constraints about that

type through the module API.

Yes, definitely.

So for example, let's say that you wanted to have a positive integer.

Elm has no way to enforce that an integer is positive because you don't have dependent

types or any fancy stuff that checks the values through the type system.

So you can say something is an int but you can't say that it's a positive int.

So how would you do that using Elm?

How would you enforce that constraint?

You would make an opaque type.

So you would create a new type positive integer.

You would put it in a different module and you would not expose the constructors.

And then you would expose one function or at least one function that would be able to

create one such positive integer.

And then one would check whether the number is positive.

And if it isn't, it doesn't return a positive integer, it returns something else.

So you could have a maybe positive integer or a result with a more precise error.

Now, okay, so this is bringing up an interesting point because there are different patterns

for using opaque types to model these constraints.

So the one you're talking about is sort of like modeling.

It's modeling a constraint through a type that represents a runtime check.

So there's this runtime check if n is greater than zero or greater than or equal to zero,

depending on how you want to define positive.

Let's just say greater than zero.

Okay, so you can check if n is greater than zero, then you give just positive integer

Otherwise, you return nothing.

That's one pattern that you can do to use a runtime check that you could validate something.

You could say, is something a valid email address and do some runtime check or things

You can do any sort of validation.

Is something a strong password?

And so if you have that type, you know that it has met the validation that you define

in that module.

And you know, the only place to look to see what the constraints are is within the boundaries

of that module.

So it makes it easier to reason about the code because you don't have to think about

every place that's calling that module.

You don't have to think about every place that might in the future call that module.

You only think about the code in that module because it's the gatekeeper for that constraint.

Yes, you don't have to think about is the age of the user above zero in the user admin

panel, for instance.

You only have to think about it in one location, which is the positive integer module.

So this is a great technique for taking a runtime check and turning it into typed data.

And there's this sort of process when you're working with Elm in general that you're always

sort of refining down a type as you go along.

You're proving more and more qualities about that data.

And so keeping track of a certain constraint or validation is really useful with a type.

You could also say, you know, this is a non empty string and you could represent that

using this sort of technique of keeping track of a runtime validation through a type that

is returned conditionally.

So you will only return that type on the condition that that validation passes.

There's another way to do it.

So let's take the positive integer example.

We talked about doing this runtime validation where we conditionally return that type.

And so if we have that type, we know that condition was met.

There's another way to do it where you don't have to return that type conditionally using

a maybe or a result.

The way you do that is you say positive integer, you define a constructor one that gives you

a positive integer one and within the API for that positive model, you know that one

is a positive int.

And then you could you could expose whatever API you want.

And of course, it depends on your domain.

It depends on the problem you're trying to solve.

But for example, you could have an increment function that increments it by one.

Now you still are guaranteeing that it's a positive int.

So you're not doing a validation at runtime that will conditionally return you a type.

You are proving through the API that it's impossible to not meet that constraint because

of the API you expose.

So you still need some validation to ensure that the integer is within acceptable bounds.

So it needs to be between zero and two billion, which is the upper limit for integers, for

So if you go past that, you still need to validate within the positive integer module

that is correct.

Yeah, I mean, that that's something that you kind of need to look at the particular needs

of your domain and figure out the way to model that.

And, you know, there's there's no one solution.

It's really an exercise in thinking about your constraints and thinking about how the

API you expose enables those constraints and rules out anything else.

So another example I think of sometimes is money.

You know, let's say you've got a dollar amount and maybe you're representing it in cents.

So I've actually run into this before where you have a dollar amount.

And if you expose the constructor, maybe maybe you really want to make sure that that money

is non negative, right?

Or maybe if you're representing that money in a negative way, maybe the internals of

how you represent that money might change over time.

So maybe you could represent it as a negative int, or you could represent that with a particular

value that says whether the money is negative or not.

And those API design decisions, those things evolve over time as your API evolves, as your

domain evolves and the problems you're trying to solve.

And so I think this leads us to another point about opaque types, which is that it allows

you to hide the implementation details and make it so that users aren't depending on

those implementation details.

And so downstream, you don't make breaking changes because you've hidden the internals

and so you can freely change them as long as the public API remains the same.

Yes, that's something that I noticed a lot of when writing applications is that you start

with the type, which is a record type, an alias to record type.

And then because the internals are exposed to the other modules, the other modules start

depending on them.

And then when you want to make changes to the API, to the internal structure, then you

get into troubles because lots of other modules depend on how that works.

And you need to, even for simple changes like adding a new field to the record, that's very

Whereas if it was a pig to start with, it would be very easy.

You only have to change it in one place, maybe also in tests and that's it.

Well, probably not even in tests.

Because the tests also depend on the public API that you've exposed.

And that's also a good point.

People often ask the question, should I test my internals with the pig types?

It's a problem solved.

Now, okay, so this leads to perhaps a more advanced topic, but I just want to put it

out there to make people aware of this.

There are some cases, so I think of these things, you can have a module opaque value

or a module opaque type, or you can have a package opaque type.

And when I say package opaque type, I'm talking about having an Elm package that you're building

and that Elm package can have a type that's internal to the package, meaning anything,

any code within that package can use that type and see the type and anything outside

of the package cannot use the internals of that type.

So there are a lot of packages that use this technique.

For example, in Elm UI, the element type is internal.

It's got some data.

It's not just a record.

If it was a record, then you wouldn't be able to hide the details of that constructor from

the outside world.

But it's some internal type that's got whatever data that custom type has.

Maybe it's got variants you don't care or know as a consumer of that type of that package.

And all of the code in that package is able to use those internal details.

And it's protecting those constraints and those validations to anybody who's using the

But it can be used anywhere.

Those internals can be directly used anywhere within that package.

Yes, because that package still enforces all the constraints on a package level.

Sometimes you use a pick types on the module level to ensure guarantees inside it.

But when you have a package and that is well built, you don't need to have them inside

You can just enforce them at the whole level, the package level.

So that could really be a whole topic for an episode.

But just to get people aware of that pattern, if you're browsing through the documentation

in an Elm package, and you see type alias element equals element, you're like, what?

It's some voodoo magic that is using this package opaque type.

So should we talk a little bit about how to organize code in opaque modules and what that

Yes, let's do that.

Okay, so this is this is a really interesting topic.

Sometimes I see people organizing their code with update helpers, view helpers.

They have like a module that has their types defined.

And I think there are times, there are plenty of times when having a module that just has

some view logic is a great idea.

There's sometimes when you might want to have some utility helpers, maybe some update function

But I think that just as a general rule, organizing your modules according to this is view related

stuff, this is update related stuff, you miss out on some of these opportunities of constraints

that you can use opaque types to help you enforce.

So for example, let's say that we had a money module, and we define the money type in one

module and we define something that shows you how to display money in another module.

Well, okay, let's say we start off with a representation of money is dollars and cents.

And we move that over to representing money as just cents, the lowest denomination.

Now since since we had a types module that had our money type, as well as several other

types for our application, that module can't be opaque because the way to present money,

the function that gives you a string form of the money, the function that gives you

an HTML form of the money, the function that allows you to add together to money values,

those things all live in separate modules.

And because they live in separate modules, you can't make it opaque, because the only

way to do an opaque type is if the type lives with the functions that deal with that type.

And so that's a general pattern that there's a type at the center of an opaque module or

just a module in general.

Now that doesn't mean that there can't be modules that just have helpers that are here

are some view helpers.

That's that's reasonable, but just be aware of, of that.

Yeah, beyond the lookout for when am I spreading the type and the things that deal with this

type in different places, I'd say that that is a code smell.

Yeah, potentially you lose some guarantees.

Because if you do that, potentially you lose some guarantees because you can use the money

type in other modules, you can use, you can look at and use the internals.

And that's not what you want to have.

So you've now started depending on the internals of that implementation throughout your entire

And so now if that changes, you have to think about, okay, was anybody depending on these

internals in a way that I'm going to break if I change them?

Whereas if you surely surely, surely no one, if I leak details, no one will use them.

I mean, you're going to get money showing up where, you know, the you're showing a hundred

times more money than you had because somebody was presenting it in some way or sending it

off to some API depending on your internals.

And you don't want to worry about that.

So for critical parts of your domain, it's, it's even more important to use these opaque

types because you want to be careful about the internals and you don't want there to

be bugs, right?

So in cases where it's very important not to have bugs, you want to hide the internals

and an opaque type of how you do that.

Well, the thing is, it's not necessarily about hiding the internals.

That's a good point.

But when you want to have the guarantees, it's not because you want to hide the details,

it's because you want to limit what people can do with it.

So you want an API to limit what are the operations that you can do with the type.

So for money, you could display it in some ways, in some formats, you could add it, you

could make it so you don't add money of different currencies, but it doesn't make sense to divide

money by money, for instance.

So you would not have the API allow that possibility.

And that's why what an opaque type allows you to do.

It limits what you can do with a type.

That's a great point.

And it also allows you to protect certain constraints about that.

So for example, I've got a blog post I wrote about this thing I use the term exit gatekeepers.

And so I like I use the example of a social security number.

So if you have a social security number, you want to be very careful with what you do with

If you have a sort of logging service, if you get some sort of error log and it just

takes some data and presents it and if it has access to the internal representation

of that type, you could imagine that the error logs are going to show up with people's social

security numbers, which is not great.

For their sake, don't do it.

The take module is a great way to be confident about protecting constraints.

And in cases of security, you want to be confident about protecting certain constraints.

In the case of things related to money, you want to be confident about protecting certain

So I wanted to touch on one point, which is I think that sometimes it makes people uncomfortable

to organize a module that has like, okay, how do I present this money as a string?

What's the HTML look like for this money?

How do I perform an HTTP request and encode this?

So like, so you might have a JSON decoder for money.

You might have a JSON encoder for money.

You might have a function that returns HTML or that returns a string.

In addition to just the functions that add them together.

I think that that can make some people uncomfortable because it feels like that's a different concern.

Like that's, that's how you display money.

That's how you turn it into HTML.

The thing is, I find that when I'm trying to understand what's going on with the code,

I often like to have it organized in that way.

It's very natural to say something's off with how this money is being formatted.

Where do I look for that?

The money module.

It's very natural, but it also has this added benefit that you can enforce all the constraints

in this one place where you have access to the internals and you have the responsibility

to enforce those constraints.

And that responsibility doesn't leak outside of that module.

So if anything is wrong with the constraints being enforced or the, or the way that that's

being presented, you know where to look.

I don't know if I would put the view inside the money module.

The reason why I would probably do that is if the view needs to contain something that

I do not want to be used somewhere else, then I would probably do that.

Like for instance, if I, if there was some, if there were some internal details that I

needed to show, but that would be dangerous to leak out, then I would make the view be

inside the money module.

Otherwise I think it's fine to have it outside the money module.

You're saying that you would define the two string function for the money inside the money

module and then anything that wanted to present the HTML would call the two string.

What if you have some special way to display it where you know you display like, I don't

know, you have some special HTML class that is applied.

So if it's negative, then you show it in red with parentheses around it, or if it's positive,

you show it in green.

And if it's zero, you show it in gray.

I guess you could create a new custom type.

Yeah, that's a good, that's a good way to approach that problem.

I guess the, the general sensibility is what you have to develop, which is I'm trying to

protect these constraints.

I don't want to leak these constraints outside of the module.

And so yeah, as you say, you can create a new custom type, which is the public interface

that describes is this, you know, maybe it's a custom type with three variants.

So it's money that's either positive, zero or negative.

And you ask for money and it gives you this custom type.

And then you have another module, or maybe you can depend on those details of those variants

of that custom type in directly in your view function or wherever it's appropriate.

But I guess the general thought process is what constraints do I want to protect?

What are the internals that I want to use to enforce those constraints and that I want

to protect the outside world from depending on?

So as long as you're doing that, that's what leads you in the right direction.

And it may take many different forms, but I do find that often the things that belong

in a particular module might be more than what somebody would expect.

So for example, how do you make an HTTP request to, you know, maybe you need to fetch an auth

token and you've got, you know, you might think like, well, I've got my HTTP requests

module and this performs all my HTTP requests.

So here's how I do an HTTP, like here are all the HTTP requests that I do to my server

and I store it in this module or I store it in the module for this page or something.

Well, in a way the details about how you perform the authentication request and how you decode

that value, they kind of belong with that authentication token type.

So there's this type, it's opaque.

You don't want somebody to be able to just construct an auth token with a string to just

fake it and pass in an auth token.

Like you're either authenticated or you're not.

If you're not authenticated, just be honest with me, you're not authenticated and I'll

let you access what you can, not being authenticated, but don't just give me an empty string.

And so the only way you can get an auth token type is by performing an HTTP request, which

is exposed from that auth token module.

And maybe that one needs to be configured, but just the function that will make that

HTTP request will take the needed parameters.

So for example, perhaps you inject the dependency from the outside of what is the base URL for

So if it's like a staging server or a dev server or a production server, you inject

So those details are not coupled to your auth token module.

So I think that this is, I just want people to be aware that it's okay to have these different

types within a module kind of living together.

It can actually work quite elegantly and just do what it takes to protect your constraints.

That's the first and foremost thing.

And nice design will follow from protecting those constraints.

I think that's a question always with when you write an Elm type is what constraints

So when you try to, when you start to leak out details, what guarantees do you lose?

So I think that, I think that if somebody wants to get started applying this technique

of using opaque types, I think that the very first thing to start doing is to start with

that mindset of thinking about the constraints and actually stepping back from that even,

what is the problem you want to solve?

And how could a type or an API and hiding some of those types in an opaque module, in

an opaque type, how would that help you enforce those constraints or solve that problem?

So any parting wisdom on how people can get started using opaque types?

I've got one very good piece of advice.

Put your type into a separate module and make it opaque.

And you can, you can take an existing module.

Let's say that you've taken, you've had a money module that exposed the constructor.

Well the good news is with Elm, I don't know if you know this, but if you start breaking

things, then the compiler will tell you what you broke.

It's a pretty cool feature.

It's like a linter or something.

It's really powerful.

So if you want to get started, a good first step is to take a type, maybe it's like a

business critical type, like money or something that's core to your domain.

Maybe it's something you've been having bugs in this area.

I had an example in Elm GraphQL in the generator code for the CLI where I generate the Elm

I had some bugs that I, bugs kept coming up around the way I was normalizing values where

in some cases I need to normalize something to be a valid Elm name so I can have an Elm

function that's based on some value from a GraphQL schema.

So sometimes I need to normalize it, but sometimes I need to use the actual name as far as the

GraphQL schema is concerned.

For example, if I'm performing the request, I need to use the name GraphQL knows, the

unnormalized name.

If I'm generating a function or a type, I need to normalize it according to certain

So I had bugs coming up in this area a few times and I'm like, you know what?

I need to use the type system and module design to protect these constraints.

Because in both cases you were using strings, I'm guessing.

The strings were getting passed down.

Exactly, so I just wrapped it in a custom type.

I made that custom type hidden in a module so it's a private constructor so it's an opaque

You know, I think I called it like a camel cased name.

So then I know this represents a camel cased name and I can get, I had basically a getter

that allows me to either get the raw unnormalized value or a valid Elm identifier.

And then what happened is I, so this is a trick that I like to use is to, you want to

hold on to the type that represents like the purest form and the constraints as long as

So I think of this as wrap early, unwrap late.

So as soon as something comes in, you want to wrap the type in this type that represents

something or as we kind of talked about, in some cases it may perform a validation and

it may conditionally return that if that validation succeeds.

Or in some cases like an auth token, you have an HTTP request and the only way to get that

auth token type is through the HTTP request function that's exposed in that module.

And so you want at the earliest possible moment, you want something to have this type which

represents the constraints about it.

And then at the latest possible moment, you want to turn that into a primitive or a side

effect or whatever it may be.

So in the case of a normalized or unnormalized name in my Elm GraphQL CLI, at the very last

possible moment, when I turn something into a GraphQL request or when I turn something

into a function name that I'm going to generate in the generated code, that's the moment that

I do it and no sooner because otherwise I'm just passing strings around.

I don't know what they represent.

I can't use it to enforce those constraints and those semantics.

And you got the same problem over again.

So now that brings up another thing which is that opaque types are a great way to enforce

Just passing something around, you know what it represents.

And as you said before, you know what the valid operations are to do with it.

So the API design, that's the only thing that allows you to protect what can I do with this

Because often what people do is they make a type alias.

Like type alias, user equals some records with a lot of details.

So the semantics are there, but you have no guarantees.

So if you want to get started with opaque types, then pick a module, pick something

that's been causing problems, creating bugs, or just been difficult to maintain.

Maybe you have to make breaking changes often because some of your code downstream that

depends on it is directly using internals.

And then you can just try one small step.

You can just make that constructor private, see what happens, see what breaks, and then

try changing those areas that break.

And then in fact, you can even expose the constructor directly as a function.

At least to start with.

At least to start with.

That's one small step that gets you in the right direction and then start playing around

with it and massaging in those constraints.

And while you do that, you will notice you will need access to certain details of the

type and some of them may look odd.

Like why do you need that information at that location?

And maybe you'll notice some smells with that.

So yeah, get good at noticing those smells.

Get good at noticing when you're depending on internals, when you're using a type that's

defined in one module, using functions defined in another module.

Be on the lookout for those types of smells.

Be on the lookout for primitive obsession, having these primitives that you're depending

on everywhere and you lose track of the semantics and the valid operations.

So you're performing operations on them that shouldn't be valid, but they're just a primitive,

so there's nothing keeping you from doing that.

I'd love to learn more about that technique.

Just get used to doing it myself.

I actually gave a talk at a conference last year that I showed this technique for, I called

it this technique of wrapping something in a semantic type.

And it's actually quite cool because you can wrap something in a type, kind of like I described

for this type that represents a value that represents a name in a GraphQL schema.

You wrap it in that type and then instead of having primitives, a string that you're

passing to this function, passing to this function, then returning in this function,

you just say, oh, actually this is no longer a string.

This is this opaque type.

And then you change all these type signatures everywhere.

Instead of saying it takes a string and it returns this, you say it takes this opaque

type and then it returns this.

And you keep doing that all the way until you get to the very end.

And then at the very end, you call some function that takes that opaque type and gives you

And all the way through, all you've done is change some type annotations so that it says,

instead of passing a primitive, you're passing an opaque type.

And then at the very end, at the very edge, you return that primitive at the last possible

You wrap early and you unwrap late.

It's very easy.

It's a very safe refactoring.

And it's a great way to have nice semantics that ensure that you're using things with

the appropriate actions.

It's very safe.

Because you only need some wiring.

You don't change business logic.

I mean, really it's the kind of thing that IDE tooling would be really good at because

it's a very tedious, mindless wiring exercise.

But it's powerful because it opens up these constraints that you can model through the

So one other way of starting with opaque types is to create a new module, new opaque type.

I think we would both advise people to start making the types opaque by default.

It's a really interesting question and it's one I've thought about.

I think at the very least, play around with it, try it, see how it feels.

Try making some things opaque by default.

You know, I mean, there are times when using a record is appropriate, right?

I mean, can your model in your application be a record?

Yeah, that's absolutely fine.

That's a great representation for that, for example.

So just I would say that try that.

Try making something opaque as a starting point.

Try creating a new module for something you're trying to represent.

I think that it does take some judgment and I think that you, because if you try to make

everything an opaque type, it does introduce an overhead.

And I think that for you saying just use an opaque type right away, I think that you have

the judgment and the experience to know this is probably going to be a good idea.

So it feels natural.

But I think that if somebody who's new to opaque modules were to try to do that, they

might say, oh, OK, well, here's my model for my application.

So I'll put that in a module and I'll make that an opaque type or I'll make that a custom

Well, there are two reasons why I would do it.

Is one, it's much easier to turn an opaque type into a non opaque type than the other

So if you have a type and you need to make it opaque, that will be tedious.

So as you said, an IDE would be great for that, but we don't have any IDE that does

Crossing my fingers.

Crossing my fingers.

So I think that what I would recommend is that push yourself out of your comfort zone

So if it feels like a stretch to start with an opaque module, maybe don't do it all the

time, always, but try doing it for a day and see how that goes and see what you learn.

When was it not working well and why was that not working well and when did it work surprisingly

You can't learn something new unless you push yourself to try a different way of working.

And with opaque modules, it's a different mindset.

And so you have to push yourself to try something new and think of it as an experiment, see

Film is great for refactoring and changing things and telling you where you need to fix

So just try something.

And then above all, experiment to try new ways of working with that mindset of what

problem am I trying to solve and how can types help me do that?

How can I enforce constraints using types and module design, API design?

Basically, if you don't gain any guarantees from using a big type, don't use it.

That's my baseline.

But as you said, you probably need some experience with that.

So try making it opaque to start with.

Don't mind the boilerplate.

When you're used to boilerplates in Elm, it feels all for us first.

It's not that bad.

And when you use boilerplates, it's usually for something good like getting new guarantees.

And one micro tip, when you're doing this technique, I think a lot of people don't know

this trick that if you have a custom type with a single variant, you can destructure

that custom type variant directly in your function signature.

So if I have type money equals money int where int is the sense.

In the money module, I can have a two string function that takes money and returns a string.

So I could say two string and then in parentheses, I can do capital money.

So that's destructuring my money just like I would do in a case statement.

But it's not in a case statement.

It's two string parentheses money.

And then I destructure that int in some variable name money value.

And now I don't have to do a case statement.

It's just one little trick that will make your life a little bit easier.

Yeah, I think we're all very happy when we learn that technique.

Yeah, absolutely.

Yeah, this single variant custom type ends up happening a lot when you're using opaque

Yeah, it removes like half of the boilerplate.

Well, I think there's a lot more to discuss in the episode, but hopefully this is a good

starting point.

Yeah, we hope you understand the big types now and that you won't be lost when we talk

about them in future episodes.

Yeah, hopefully if people just give these a try, take them for a spin and start taking

a step in this direction of experimenting, then that's a success.

So take them for a spin.

See what you think.

All right, Jeroen.

Well, I'll talk to you next time.

Have a good day.

Have a good day.