In many (but not all) scenarios "Make illegal states unrepresentable" is way too expensive to implement.
Especially when dealing with a fast changing domain, having to support different versions of data shapes across long time periods: dynamic data definitions are more economic and will still provide sufficient runtime protection.
"Errors as values" - what is an error? I see this pattern misused often, because not enough thought was put into the definition of an error.
"Disk is Full" vs. "data input violates some business rule" are two very - very - different things and should be treated differently. Exceptions are the right abstraction in the first case. It's not in the second case.
""Make illegal states unrepresentable" is way too expensive to implement."
This has not been my experience. The speed increase in development not having to worry about the unrepresentable cases have been very valuable. In addition as requirements change migrating old data hasn't been a huge concern. For code changes refactoring the types helps address new cases as well.
These are great ideas and patterns even if you’re not doing functional programming.
FP-first/only languages tend to push you in these directions because it makes programming with them easier.
In languages where FP is optional, it takes discipline and sometimes charisma to follow these affirmations/patterns/principles.. but they’re worth it IMO.
Maybe not in literally every language, but, to cherry pick some examples:
Java (along with many other object-oriented languages) lets you create objects that are effectively immutable by declaring all fields private and not providing any property setters or other methods that would mutate the state.
Errors as values is one of the headline features of both Go and Rust, neither of which has do notation and monads.
Functional core, imperative shell is something I first learned in C#, but I don't think it was _really_ at significantly more of a disadvantage than most other languages. The only ones that let you really enforce "functional core, imperative shell" with strong language support are the pure functional languages. But people working in, say, Ocaml or Scala somehow still survive. And they do it using the same technique that people working in Java would: code review and discipline.
None of this is to say that language-level support to make it easier to stick to these principles is not valuable. But treating it as if it were a lost cause when you don't have those features in the language is the epitome of making the perfect the enemy of the good.
Coincidentally, "functional core, imperative shell" is how most companies get to adopt F# from what I was told (and had seen). It integrates really well. C# itself is a proper multi-paradigm language nowadays, and so is also quite successful at employing functional constructs.
> How can you "Make illegal states unrepresentable" with mutable state and sequences of mutations that cannot be enforced with the type system?
I think you're confusing "make illegal states unrepresentable" with "parse, don't verify"? If your type cannot represent any invalid states, there's no way you can reach them through mutation.
> How can you do "Errors as values" at a large scale without do-notation / monads?
You don't need monads for this. You just need the ability to encode your error in some term like `Either e a` and ability to eliminate those terms. In Rust for example that is the `Result` type and you use pattern matching to eliminate those terms.
That's merely syntax sugar. Haskell doesn't even have this sugar and so in a monad you have to bind explicitly, and it's fine. It's not a pyramid of doom.
Let's consider a language like C++ where it has none of those syntax sugars. The absl::StatusOr is a class with the idea that errors should be values. The standard library has std::expected since C++23. So where is your pyramid of doom?
Genuinely curious for all these follow up questions:
Is immutability exclusive to functional programming?
Is the ability to use data/values exclusive to functional programming?
Are monads exclusive to functional programming?
For discussions like this, how do we separate "it was done first in functional programming but can also be done in procedural programming" with "it cannot be followed outside of functional programming"?
> Is immutability exclusive to functional programming?
No, but immutable defaults are powerful.
E.g. in JavaScript / Python, the built-in lists and dictionarys (which are blessed with special syntax) are mutable.
> Is the ability to use data/values exclusive to functional programming?
No, but expression-orientation makes this less painful
> Are monads exclusive to functional programming?
You can hack them in by abusing co-routines or perhaps async/await in various languages, but it will never be as good as something built for this purpose.
Type-inferences, type-classes and do-notation make monads workable in practice.
You don't need coroutines or async or anything complicated to model monads, just functions and data structures. Search for "c++ monads" and you'll find a ton of examples.
As someone who's written a pure functional framework for C# [1], I'll bite...
> How can you "Make illegal states unrepresentable" with mutable state and sequences of mutations that cannot be enforced with the type system?
Firstly, don't use mutable state, write immutable types. Secondly, write constructors that reject poorly formed data structures. Thirdly, for existing libraries with types that are mutable, create a wrapper for the library with functions that return an IO/effect monad.
> How can you do "Errors as values" at a large scale without do-notation / monads?
Luckily, from a C# PoV, we have LINQ, which are monads. I agree that manual management of monadic flow would be hard without something akin to do-notation or LINQ.
You can get quite far with fluent methods, but a general monadic-bind is quite hard to chain if you want to carry all of the extracted values through to subsequent expressions (lots of nesting), so yeah, it would not be ideal in those languages. It should be stated that plenty of functional languages also don't have do-notation equivalents though.
> How can you do "Functional core, imperative shell" without the ability to create mini DSLs and interpreters in your language?
I've never really liked the "Functional core, imperative shell" thing. I think it's an admission that you're going to give up trying to be functional when it gets difficult (i.e. interacting with the real world). It is entirely possible to be functional all the way through a code-base.
In terms of DSLs: I'm not sure I know any language that can't implement a DSL and interpreter. Most people don't realise that the Gang of Four Interpreter pattern is isomorphic to free-monads, so most imperative languages have the ability to do the equivalent of free-monads.
As the GP states, it takes discipline to stick to the constraints that a language like Haskell imposes by default. Not sure about the charisma part!
I have found that having access to a world-class compiler, tooling, and large ecosystem to be more valuable to getting shit done than the exact language you choose. So, bringing the benefits of the pure-FP world into the place where I can get shit done is better than switching to, say Haskell, where it's harder to get shit done due to ecosystem limitations.
There's also the get out of jail free card, which allows me to do some gnarly high-performance code in an imperative way. And, as long as I wrap it up in a function that acts in a referentially transparent way, then I can still compose it with the rest of my pure code without concern. I just need to be a bit more careful when I do that and make sure it's for the right reasons (i.e. later stage optimisations). That's less easy to do in FP languages.
Again, it's about discipline.
If you want to see how this can look in a mainstream, imperative-first, language. I have a few samples in the repo, the one I like to share when helping OO-peeps learn about monads is this game of 21/Pontoon [2]. I suspect most people won't have seen C# look like this!
Software developed on a team where everyone has different values/principles... some times it's not the technical discipline that is required, it's convincing folks to adopt these patterns and stick with them that's the hard lift. :)
> To me, this simply makes more sense: isn’t it objectively better to get a finite and predictable error value from a function than an unspecified exception that may or may not happen that you still have to guard against?
Whether an error is returned as a value or thrown is orthogonal to whether it is finite and predictable. Java has checked exceptions. In Swift you also can specify the exceptions that a function may throw. How is it any less predictable than return values?
Semantically, a thrown exception is simply a return value with debug information that gets automatically returned by the caller unless specified otherwise. It is simply a very handy way to reduce boilerplate. Isn't it objectively better to not write the same thing over and over again?
I agree with regards to checked exceptions. Unfortunately Java doesn't support any form of polymorphism over thrown exceptions so it makes your code much harder to reuse. In languages that support polymorphic effects I imagine this is less of a concern.
I do not work in a functional language, but these ideas have helped me a lot anyway. The only idea here that I find less directly applicable outside purely functional languages is the "Errors as values [instead of exceptions]" one.
On the surface, it makes complete sense, the non-locality of exceptions make them hard to reason about for the same reasons that GOTO is hard to reason about, and representing failure modes by values completely eliminates this non-locality. And in purely functional languages, that's the end of the story. But in imperative languages, we can do something like this:
def my_effectful_function():
if the_thing_is_bad:
# do the failure thing
raise Exception
# or
return Failure()
return Success()
and a client of this function might do something like this:
and completely ignore the failure case. Now, ignoring the failure is possible with both the exception and the failure value, but in the case of the failure value, it's much more likely to go unnoticed. The exception version is much more in line with the "let it crash" philosophy of Erlang and Elixir, and I'm not sure if the benefits of locality outweigh those of the "let it crash" philosophy.
Have any imperative folks here successfully used the "errors as values" idea?
Rust does "errors as values" pretty well, see [1]. You can manually handle them if you want, or just apply the '?' operator to auto-propagate them out of the function. In both cases, it is obvious that there was an error handling needed.
Non-locality of exceptions is a feature, not a bug. It's so you can focus on the success case, instead of error case, when reading your code. It's usually, but not always, what you want. "errors as values" is effectively the same thing as exceptions anyway, except it's explicit - meaning it's hurting readability by default and adding extra work[0]; modern languages go to extreme length to try and paper it over with syntactic magic.
Unfortunately, the whole "exceptions vs. expected" issue is fundamentally unsolvable for as long as we stick to working on common single source of truth plaintext code. Explicit and implicit error handling are both useful in different circumstances - it's entirely a presentation issue (i.e. how code looks to you); our current paradigm forces us to precommit to one or the other, which just plain sucks.
--
[0] - See how it works in C++, where you can't easily hide the mechanism.
In general, I find that explicit code is more easily read than implicit code. I prefer static over dynamic typing, I actually _like_ the explicitness of async/await or the IO monad. If something allows me to find out information about my current context without having to move up or down the stack and reading the code in other functions, I'm pretty happy about that something, because reading code is slow and tedious. What is it about implicit code that makes you feel it's more readable?
The way to do functional programming in imperative languages is to handle the side-effects as high up in the call-chain as possible. That would mean that you return an instance of Error from lower-level and decide in some higher caller what to do about it.
That as an alternative to throwing the error. This way you get the benefit of being able to follow the flow of control from each called function back to each caller, as opposed to control jumping around wildly because of thrown errors.
In a statically typed imperative language that would need support for sum-types, to be able to return either an error or a non-error-value. Then you would be less likely to ignore the errors by accident because you would always see the return value is maybe an error.
Isn't this also basically how Haskell does it, handling side-effectful values as high up as as possible which in Haskell means moving them into the runtime system above all user-code?
Right, I understand. But my question is, how do you _ensure_ a failure value is dealt with by clients? In purely functional languages, your clients have no choice, they'll have to do something with it. In imperative languages, they can just ignore it.
This is a nice ideal to shoot for, but strict adherence as advocated in the article is a short path to algorithmic explosions and unusable interfaces on real life systems.
For example, if you have two options that are mutually incompatible, this principle says you don't make them booleans, but instead a strict enum type populated with only legal combinations of the options. A great idea until you have 20 options to account for and your enum is now 2^16 entries long. Then your company opens a branch in a different country with a different regulatory framework and the options list grows to 50 and you code no longer fits on a hard drive.
If you truly do have 2^16 valid and distinct behaviors, it is not possible for humans to correctly write 2^16 different code paths anyway.
More than likely, the majority of the combinations of your 29 Boolean flags are invalid. You should strive to have that checked by the program, and not only as a note in external documentation.
No one is saying int should be turned into an enum.
You only have 20 options, but making that many distinctive options is not exactly a stretch. It's not like every single set of options is its own code path, most options represents a small deviation at one particular part of the code. Most of the options aren't mutually exclusive either, only a few combinations are illegal.
Imagine a simple shipping system for example. The package might be routed via truck, boat, plane, pipeline, foot, etc... Probably even a combination of those options. The package might be low, medium, or high priority, although high priority packages are not allowed to be delivered by boat. The package might be small, large, or liquid, but liquids can't be delivered by foot. There are 195 different countries with even more regulatory regimes to consider, some of which may have different customs requirements based on mode of transport. Now imagine a problem that is actually complicated.
The idea of encoding all of this business logic into the datatype is a road to madness IMHO. Especially if the rules can change on you and require you to rework your datatypes to match. On the small scale this makes a lot of sense and is good advice, but strict adherence is impractical.
You don't need a single type to represent the entire program state.
We probably both agree that separate types for shipping methods, priorities, size, country makes sense.
The API can be designed to prevent illegal transitions between types to arrive at an invalid state. The exact implementation depends on the language.
> The idea of encoding all of this business logic into the datatype is a road to madness IMHO
The alternative is hoping that every developer remembers not to violate any of the implicit and unenforced rules.
If the system is too complicated to be represented in code, can a human realistically hold the entire state machine in their head as they make changes?
The parent is discussing a situation where you have 16 distinct, binary states many but not all of which are mutually compatible. So you can have 16 bit vector of the states, 16 binary variables or an enum of valid states - the enum would have O(2^16) members because of the distribution of valid states but unlike the others, an invalid state would not be possible to represent.
Nobody says you have to have a single enum type containing all the combinations. Chances are, you can use sum types (discriminated unions) to factor things nicely if you think about them. For example if option B is only relevant when option A is set to true, you can have something like
data OptA = ATrue OptB | AFalse
data OptB = BTrue | BFalse
There are three valid combinations but no type has three alternatives. Nobody in their right mind would write out an enum with 2^16 cases. If you do, you are misusing enums and it's time to consider other tools in your language.
Nobody says you have to have a single enum type containing all the combinations.
No, no one would continue up to 2^16 and the code would get unmanageable long before that. But it's illustration of the problems starting out dealing with the invalid states of two variables using an enum because what happens when more and more variables arrive? Sure, the standard answer is "just refactor" but my experience is no client or boss wants to hear "adding this small state is going require a lot of change" and a trickle of binary conditions is a very common occurrence as is code expanding to handle these (and become excessively complicated).
Chances are, you can use sum types (discriminated unions) to factor things nicely if you think about them.
Maybe you have a good chance of combining these binary conditions in a good way. But I mention you've substituted a hard problem instance (factoring binary conditions) for an easy problem instance (checking binary conditions). Functional programming has a weird dual personality where on the one hand you hear "A functional programmer is always a smarty and solve hard problems as a matter of course" but also you hear "functional programming would be the dominant paradigm if only ... we taught people young so they wouldn't have their bad procedural habits"
Imagine a case where you have 4 options. W, X, Y, Z.
Y and Z are mutually exclusive.
X can only be set if W is set.
If Y is set then X must be set.
Going down this road you end up encoding your business logic into your datatypes. Which is good to a degree, but makes things messy when new options are added or requirements change. Imagine a new option U is introduced that is only valid when W is unset and Z is set but allows X to be set. Your interface becomes very hard to manage with even a small amount of complexity added to the options.
True. And what is an "illegal state" anyway? If two options are "mutually incompatible" it means that they, as far as we know, so far, have never occured at the same time.
But that is just how things are currently. The world may change. And it's important to strive for maintainable code that can accommodate such change easily.
Expanding company operations to work in a different country is an example of this (our) "world changing". States that never occur together here, may occur together there. Or in the future more generally.
So, making illegal states non-representable is about avoiding errors in respect to the system specification. But it does not take into account the fact that in the real world specifications typically evolve and change even during development, and perhaps more so later.
This is technically correct but disingenuous. This reminds of the climate change comic where a scientist asks “What if climate change is a big hoax and we create a better world god nothing?”
This is an excellent illustration. I feel functional programming provides great tools and concept for a cohesive system (one with consistent requirements, which accomplishes a single thing, etc). But many programs you write involve parts that aren't very cohesive and trying to make them cohesive is far more work than just bolting on a case statement or similar things.
Parent currently downvoted without comments. Seems like a sad way to respond.
I feel like you might be missing the point. A case statement is not something you bolt on as a hack. It is the right tool for the job the vast majority of the times. When you use case statements, you refine and reduce your state space. It makes code easier to understand. When you combine this with the idea of making illegal states unrepresentable, a case statement gives you an exhaustive listing of what could happen. Even a lot of Haskell programmers, after using things like the `maybe` function to eliminate Maybe types and things like `fmap` over Maybe, eventually find that using case expressions produces the clearest code.
I really hope HN enforces a rule that a downvote must be accompanied by a reply.
If you only have 3 states, then yes, that should be an enum, not a pair of booleans, because you have 3 states, not 2x2 independent ones. Making the 4th state unrepresentable removes code and error checking. It's also just simple domain modeling.
Your latter example needs context. In what situation have you had an enum with 2^16 states? In any case, if you generally have a reasonable number of booleans, with some states being logically invalid, then you'll need error checking on those anyway.
Leaving them as booleans gives you the ability to monkey-patch things later in an ad-hoc manner, which is useful when you're still in a prototyping phase and the requirements aren't clear (and you could argue that this is almost always the case in many problem domains; I think that would be valid criticism.) But if you've already modeled the domain and you want any assurance of correctness, then I don't see why you wouldn't enforce the constraints at the type level. Your criticism seems to me the usual trade-off between strong static typing and dynamic and/or weak monkey-typing.
Imagine you have a program where there are 4 options, W, X, Y, Z. Y and Z can't be set at the same time, and X can't be set unless W is set. If Y is set then X must be set as well.
How do you represent this in a way that makes it impossible, even through programmer error elsewhere in the program, to have the flags in an invalid state?
You can create en enum that looks like:
enum program_state =
(
W_X_Y_NZ,
W_NX_NY_Z,
NW_NX_NY_Z,
... and so on
);
FP nerd: The pure core is nice and composable, with the imperative shell at the boundary.
State Skeptic: Yes. But! How do you compose the 'pure core + impure shell' pieces?
FPN: Obviously, you compose the pure pieces separately. Your app can be built using libraries built from libraries.... And, then build the imperative shell separately.
My take is that the above solution is not so easy. (atleast to me!) (and not easy for both FP and non-FP systems).
Take an example like GUI components. Ideally, you should be able to compose several components into a single component (culminating in the app) and not have a custom implementation of a giant state store which is kept in something like Redux and you define the views and modifiers using this store.
Say, you have a bunch of UI components each given as a view computed as a function from a value and possible UI events which can either modify the value, remain unhandled or configurable as either. Ex: dialog box which handles text events but leaves the 'OK' submission to the container.
There are atleast two different kinds of composability (cue quote in SICP Ch1 by Locke) - aggregation and abstraction. Ex: Having a sequence of text inputs in the document(aggregation) and then abstracting to a list of distances between cities. This abstraction can puts constraints on values of the parts, both individually and across parts. There is also extension/enrichment, the dual of abstraction.
This larger abstracted component itself is now a view dependent on a value and more abstract events. But, composing recursively leads to state being held in multiple layers and computations repeated across layers. This is somewhat ameliorated by sharing of immutable parts and react like reconciliation. But, you have to express your top->down functions incrementally which is not trivial.
> But, composing recursively leads to state being held in multiple layers and computations repeated across layers.
True, which is why re-frame has a dependency graph and subscriptions that avoid re-computation, i.e. the data dependencies are outside any view tree.
If data changes, only active nodes (ones that have been subscribed to) will re-compute. If nothing changed in a node, any dependent nodes will not re-compute.
FP is not a silver bullet. GUI is the classic OOP showcase.
> Ideally, you should be able compose them several of them into a single app and not have a custom implementation of a giant state
If you are suggesting that components store their state, I'm not sure about "ideal" there. That works well for small GUI applications. In GUI applications of modest size, you do want a separate, well-organized and non-redundant data layer you can make sense of, at least from my experience. Qt, specifically, allows you to do both things.
To your main point, I wouldn't say exactly that the component stores the state. But, rather that every component provides an initial value, possible events, and a default event handler which is a function from value to value. In effect, this is partially 'storing local state', but the above pieces can be composed to create a container component.
Note that there is no option really - the app wont be reimplementing how a key is handled in a text box. But composability means that the same principle should hold not just for OS/browser components but also for higher level components (A custom map or a tree-view where there are restrictions on types and number of nodes - these should also have default handling and delegation to upper levels.)
The global store choice makes it harder to have component libraries. But, the composable alternative has its problems too - redundancy and communication which skips layers (which requires 'context' in React).
This is a digression, but regarding OOP, my somewhat provocative view, is that it is not a natural thing, but in most languages, it is atleast 4 different concepts 1. Encapsulation/Namespace, 2. Polymorphism, 3. Extensibility(Inheritance is a special case) 4.Mutability.
These four concepts are forced/complected into a 'class' construct, but they need not be.
In particular, FP only varies on 4, but languages like ML,Clojure do 1,2,3 even better than OOP languages. Modules for encapsulation, Dispatch on first or even all arguments for polymorphism and first class modules, ML style, for extensibility.
Aside: There was a recent post (https://osa1.net/posts/2024-10-09-oop-good.html) (by someone who worked on GHC no less), favorably comparing how OOP does extensibility to Haskell typeclasses, which are not first class, but modules in ML languages can do what he wants and in a much more flexible way than inheritance!
There is also the dynamic aspect of orginal OOP - message passing instead of method invocation, but this is about dynamic vs static rather than OOP vs FP.
What OOP languages have managed to do which static FP hasn't done yet is the amazing live inspectable environments which lead to iterable development like we see in Smalltalk. The challenge is to do this in a more FP way while being modular.
This page (https://reasonml.github.io/docs/en/module) is useful to see how a FP language can do what he wants. Because we have functors, which are functions from a group of modules/classes to another module/class, we can have Composition, Inheritance(single/multiple), Mixins etc.
One way to achieve "Make illegal states unrepresentable" is by using "refined" types, a.k.a. highly constrained types. There is a "refined" library in both Haskell and Scala and the "iron" library for Scala 3.
It's hard not to giggle when the conclusion right after "Smart constructors" says "Do these ideas belong only in functional programming? While they are practiced more there...".
Ah yes, because using constructors to ensure that new objects are in a valid state is virtually unheard of in object-oriented programming.
One thing this article does is assume extreme functional mindset, I dont even think OOP enters into the authors mind- With that context, I think that statement isn't about object constructors but type constructors.
> This is fine but it's just a rehash of old well-knowned stuff.
A significant share of the current dev community wasn't there when these principles were first described. Sometimes, bringing up old topics once again helps seeing that many people had never heard of them yet.
Do you have an example of the many books and established blogs that you can share? A newcomer might not be familiar with where to start when attempting to learn more about functional programming.
~~I can't tell if you're being sarcastic, but that's exactly what TFA does. But just to pull it out explicitly:~~
Edit: I think I misunderstood your point. You were asking for a similar kind of list as TFA from the other commenter which may not necessarily be the *same* 5 ideas
In many (but not all) scenarios "Make illegal states unrepresentable" is way too expensive to implement.
Especially when dealing with a fast changing domain, having to support different versions of data shapes across long time periods: dynamic data definitions are more economic and will still provide sufficient runtime protection.
"Errors as values" - what is an error? I see this pattern misused often, because not enough thought was put into the definition of an error.
"Disk is Full" vs. "data input violates some business rule" are two very - very - different things and should be treated differently. Exceptions are the right abstraction in the first case. It's not in the second case.
"Functional core, imperative shell" - 100% agreement here.
""Make illegal states unrepresentable" is way too expensive to implement."
This has not been my experience. The speed increase in development not having to worry about the unrepresentable cases have been very valuable. In addition as requirements change migrating old data hasn't been a huge concern. For code changes refactoring the types helps address new cases as well.
These are great ideas and patterns even if you’re not doing functional programming.
FP-first/only languages tend to push you in these directions because it makes programming with them easier.
In languages where FP is optional, it takes discipline and sometimes charisma to follow these affirmations/patterns/principles.. but they’re worth it IMO.
I'm not convinced that you can follow all of these outside of Functional Programming.
How can you "Make illegal states unrepresentable" with mutable state and sequences of mutations that cannot be enforced with the type system?
How can you do "Errors as values" at a large scale without do-notation / monads?
How can you do "Functional core, imperative shell" without the ability to create mini DSLs and interpreters in your language?
Maybe not in literally every language, but, to cherry pick some examples:
Java (along with many other object-oriented languages) lets you create objects that are effectively immutable by declaring all fields private and not providing any property setters or other methods that would mutate the state.
Errors as values is one of the headline features of both Go and Rust, neither of which has do notation and monads.
Functional core, imperative shell is something I first learned in C#, but I don't think it was _really_ at significantly more of a disadvantage than most other languages. The only ones that let you really enforce "functional core, imperative shell" with strong language support are the pure functional languages. But people working in, say, Ocaml or Scala somehow still survive. And they do it using the same technique that people working in Java would: code review and discipline.
None of this is to say that language-level support to make it easier to stick to these principles is not valuable. But treating it as if it were a lost cause when you don't have those features in the language is the epitome of making the perfect the enemy of the good.
Coincidentally, "functional core, imperative shell" is how most companies get to adopt F# from what I was told (and had seen). It integrates really well. C# itself is a proper multi-paradigm language nowadays, and so is also quite successful at employing functional constructs.
> How can you "Make illegal states unrepresentable" with mutable state and sequences of mutations that cannot be enforced with the type system?
I think you're confusing "make illegal states unrepresentable" with "parse, don't verify"? If your type cannot represent any invalid states, there's no way you can reach them through mutation.
> How can you "Make illegal states unrepresentable" with mutable state
By either making the data const, or encapsulating mutable state with private fields and public methods
> How can you do "Errors as values"
Go, Rust, Odin, Zig, and many more are imperative languages that do exactly this
> How can you do "Functional core, imperative shell"
Write a function that takes the old state and returns the new state
> By either marking your data as const, or encapsulating mutations with private fields and public methods
That would seem to be making illegal states unreachable rather than unrepresentable, closer in spirit to "parse, don't verify".
> How can you do "Errors as values" at a large scale without do-notation / monads?
You don't need monads for this. You just need the ability to encode your error in some term like `Either e a` and ability to eliminate those terms. In Rust for example that is the `Result` type and you use pattern matching to eliminate those terms.
Then you end up with a pyramid of doom. Fine for small examples but it doesn't scale up easily.
Rust has special syntax (`?`) for this AFAICT.
You still don't need monads for any of this. Monads give you an ad-hoc generic way of doing monadic actions.
Short circuiting on `Either` is a specific case of this. You can define your own EitherBind in any language.
Now you can bind over Either to your heart's content without needing an encoding of Monads in your language.That's merely syntax sugar. Haskell doesn't even have this sugar and so in a monad you have to bind explicitly, and it's fine. It's not a pyramid of doom.
Haskell has do-notation which is a more general kind of this syntactic sugar
Yeah that's what I said about bind.
Let's consider a language like C++ where it has none of those syntax sugars. The absl::StatusOr is a class with the idea that errors should be values. The standard library has std::expected since C++23. So where is your pyramid of doom?
`?` and `let ... else` both partially address this, yeah.
Genuinely curious for all these follow up questions:
Is immutability exclusive to functional programming?
Is the ability to use data/values exclusive to functional programming?
Are monads exclusive to functional programming?
For discussions like this, how do we separate "it was done first in functional programming but can also be done in procedural programming" with "it cannot be followed outside of functional programming"?
> Is immutability exclusive to functional programming?
No, but immutable defaults are powerful.
E.g. in JavaScript / Python, the built-in lists and dictionarys (which are blessed with special syntax) are mutable.
> Is the ability to use data/values exclusive to functional programming?
No, but expression-orientation makes this less painful
> Are monads exclusive to functional programming?
You can hack them in by abusing co-routines or perhaps async/await in various languages, but it will never be as good as something built for this purpose.
Type-inferences, type-classes and do-notation make monads workable in practice.
You don't need coroutines or async or anything complicated to model monads, just functions and data structures. Search for "c++ monads" and you'll find a ton of examples.
You need the syntax if you want it to actually work well in practice.
As someone who's written a pure functional framework for C# [1], I'll bite...
> How can you "Make illegal states unrepresentable" with mutable state and sequences of mutations that cannot be enforced with the type system?
Firstly, don't use mutable state, write immutable types. Secondly, write constructors that reject poorly formed data structures. Thirdly, for existing libraries with types that are mutable, create a wrapper for the library with functions that return an IO/effect monad.
> How can you do "Errors as values" at a large scale without do-notation / monads?
Luckily, from a C# PoV, we have LINQ, which are monads. I agree that manual management of monadic flow would be hard without something akin to do-notation or LINQ.
You can get quite far with fluent methods, but a general monadic-bind is quite hard to chain if you want to carry all of the extracted values through to subsequent expressions (lots of nesting), so yeah, it would not be ideal in those languages. It should be stated that plenty of functional languages also don't have do-notation equivalents though.
> How can you do "Functional core, imperative shell" without the ability to create mini DSLs and interpreters in your language?
I've never really liked the "Functional core, imperative shell" thing. I think it's an admission that you're going to give up trying to be functional when it gets difficult (i.e. interacting with the real world). It is entirely possible to be functional all the way through a code-base.
In terms of DSLs: I'm not sure I know any language that can't implement a DSL and interpreter. Most people don't realise that the Gang of Four Interpreter pattern is isomorphic to free-monads, so most imperative languages have the ability to do the equivalent of free-monads.
As the GP states, it takes discipline to stick to the constraints that a language like Haskell imposes by default. Not sure about the charisma part!
I have found that having access to a world-class compiler, tooling, and large ecosystem to be more valuable to getting shit done than the exact language you choose. So, bringing the benefits of the pure-FP world into the place where I can get shit done is better than switching to, say Haskell, where it's harder to get shit done due to ecosystem limitations.
There's also the get out of jail free card, which allows me to do some gnarly high-performance code in an imperative way. And, as long as I wrap it up in a function that acts in a referentially transparent way, then I can still compose it with the rest of my pure code without concern. I just need to be a bit more careful when I do that and make sure it's for the right reasons (i.e. later stage optimisations). That's less easy to do in FP languages.
Again, it's about discipline.
If you want to see how this can look in a mainstream, imperative-first, language. I have a few samples in the repo, the one I like to share when helping OO-peeps learn about monads is this game of 21/Pontoon [2]. I suspect most people won't have seen C# look like this!
[1] https://github.com/louthy/language-ext/
[2] https://github.com/louthy/language-ext/blob/main/Samples/Car...
> Not sure about the charisma part!
Software developed on a team where everyone has different values/principles... some times it's not the technical discipline that is required, it's convincing folks to adopt these patterns and stick with them that's the hard lift. :)
Mostly functional programming does not work (https://queue.acm.org/detail.cfm?id=2611829)
> Errors as values
> To me, this simply makes more sense: isn’t it objectively better to get a finite and predictable error value from a function than an unspecified exception that may or may not happen that you still have to guard against?
Whether an error is returned as a value or thrown is orthogonal to whether it is finite and predictable. Java has checked exceptions. In Swift you also can specify the exceptions that a function may throw. How is it any less predictable than return values?
Semantically, a thrown exception is simply a return value with debug information that gets automatically returned by the caller unless specified otherwise. It is simply a very handy way to reduce boilerplate. Isn't it objectively better to not write the same thing over and over again?
I agree with regards to checked exceptions. Unfortunately Java doesn't support any form of polymorphism over thrown exceptions so it makes your code much harder to reuse. In languages that support polymorphic effects I imagine this is less of a concern.
I do not work in a functional language, but these ideas have helped me a lot anyway. The only idea here that I find less directly applicable outside purely functional languages is the "Errors as values [instead of exceptions]" one.
On the surface, it makes complete sense, the non-locality of exceptions make them hard to reason about for the same reasons that GOTO is hard to reason about, and representing failure modes by values completely eliminates this non-locality. And in purely functional languages, that's the end of the story. But in imperative languages, we can do something like this:
and a client of this function might do something like this: and completely ignore the failure case. Now, ignoring the failure is possible with both the exception and the failure value, but in the case of the failure value, it's much more likely to go unnoticed. The exception version is much more in line with the "let it crash" philosophy of Erlang and Elixir, and I'm not sure if the benefits of locality outweigh those of the "let it crash" philosophy.Have any imperative folks here successfully used the "errors as values" idea?
Rust does "errors as values" pretty well, see [1]. You can manually handle them if you want, or just apply the '?' operator to auto-propagate them out of the function. In both cases, it is obvious that there was an error handling needed.
[1] https://doc.rust-lang.org/book/ch09-02-recoverable-errors-wi...
I didn't know that! Every time I hear about Rust my opinion of it grows brighter. Thanks for sharing this!
Non-locality of exceptions is a feature, not a bug. It's so you can focus on the success case, instead of error case, when reading your code. It's usually, but not always, what you want. "errors as values" is effectively the same thing as exceptions anyway, except it's explicit - meaning it's hurting readability by default and adding extra work[0]; modern languages go to extreme length to try and paper it over with syntactic magic.
Unfortunately, the whole "exceptions vs. expected" issue is fundamentally unsolvable for as long as we stick to working on common single source of truth plaintext code. Explicit and implicit error handling are both useful in different circumstances - it's entirely a presentation issue (i.e. how code looks to you); our current paradigm forces us to precommit to one or the other, which just plain sucks.
--
[0] - See how it works in C++, where you can't easily hide the mechanism.
In general, I find that explicit code is more easily read than implicit code. I prefer static over dynamic typing, I actually _like_ the explicitness of async/await or the IO monad. If something allows me to find out information about my current context without having to move up or down the stack and reading the code in other functions, I'm pretty happy about that something, because reading code is slow and tedious. What is it about implicit code that makes you feel it's more readable?
The way to do functional programming in imperative languages is to handle the side-effects as high up in the call-chain as possible. That would mean that you return an instance of Error from lower-level and decide in some higher caller what to do about it.
That as an alternative to throwing the error. This way you get the benefit of being able to follow the flow of control from each called function back to each caller, as opposed to control jumping around wildly because of thrown errors.
In a statically typed imperative language that would need support for sum-types, to be able to return either an error or a non-error-value. Then you would be less likely to ignore the errors by accident because you would always see the return value is maybe an error.
Isn't this also basically how Haskell does it, handling side-effectful values as high up as as possible which in Haskell means moving them into the runtime system above all user-code?
Right, I understand. But my question is, how do you _ensure_ a failure value is dealt with by clients? In purely functional languages, your clients have no choice, they'll have to do something with it. In imperative languages, they can just ignore it.
> Make illegal states unrepresentable
This is a nice ideal to shoot for, but strict adherence as advocated in the article is a short path to algorithmic explosions and unusable interfaces on real life systems.
For example, if you have two options that are mutually incompatible, this principle says you don't make them booleans, but instead a strict enum type populated with only legal combinations of the options. A great idea until you have 20 options to account for and your enum is now 2^16 entries long. Then your company opens a branch in a different country with a different regulatory framework and the options list grows to 50 and you code no longer fits on a hard drive.
The mistake here is having 2^16 valid options.
If you truly do have 2^16 valid and distinct behaviors, it is not possible for humans to correctly write 2^16 different code paths anyway.
More than likely, the majority of the combinations of your 29 Boolean flags are invalid. You should strive to have that checked by the program, and not only as a note in external documentation.
No one is saying int should be turned into an enum.
You only have 20 options, but making that many distinctive options is not exactly a stretch. It's not like every single set of options is its own code path, most options represents a small deviation at one particular part of the code. Most of the options aren't mutually exclusive either, only a few combinations are illegal.
Imagine a simple shipping system for example. The package might be routed via truck, boat, plane, pipeline, foot, etc... Probably even a combination of those options. The package might be low, medium, or high priority, although high priority packages are not allowed to be delivered by boat. The package might be small, large, or liquid, but liquids can't be delivered by foot. There are 195 different countries with even more regulatory regimes to consider, some of which may have different customs requirements based on mode of transport. Now imagine a problem that is actually complicated.
The idea of encoding all of this business logic into the datatype is a road to madness IMHO. Especially if the rules can change on you and require you to rework your datatypes to match. On the small scale this makes a lot of sense and is good advice, but strict adherence is impractical.
You don't need a single type to represent the entire program state.
We probably both agree that separate types for shipping methods, priorities, size, country makes sense.
The API can be designed to prevent illegal transitions between types to arrive at an invalid state. The exact implementation depends on the language.
> The idea of encoding all of this business logic into the datatype is a road to madness IMHO
The alternative is hoping that every developer remembers not to violate any of the implicit and unenforced rules.
If the system is too complicated to be represented in code, can a human realistically hold the entire state machine in their head as they make changes?
The parent is discussing a situation where you have 16 distinct, binary states many but not all of which are mutually compatible. So you can have 16 bit vector of the states, 16 binary variables or an enum of valid states - the enum would have O(2^16) members because of the distribution of valid states but unlike the others, an invalid state would not be possible to represent.
Nobody says you have to have a single enum type containing all the combinations. Chances are, you can use sum types (discriminated unions) to factor things nicely if you think about them. For example if option B is only relevant when option A is set to true, you can have something like
There are three valid combinations but no type has three alternatives. Nobody in their right mind would write out an enum with 2^16 cases. If you do, you are misusing enums and it's time to consider other tools in your language.Nobody says you have to have a single enum type containing all the combinations.
No, no one would continue up to 2^16 and the code would get unmanageable long before that. But it's illustration of the problems starting out dealing with the invalid states of two variables using an enum because what happens when more and more variables arrive? Sure, the standard answer is "just refactor" but my experience is no client or boss wants to hear "adding this small state is going require a lot of change" and a trickle of binary conditions is a very common occurrence as is code expanding to handle these (and become excessively complicated).
Chances are, you can use sum types (discriminated unions) to factor things nicely if you think about them.
Maybe you have a good chance of combining these binary conditions in a good way. But I mention you've substituted a hard problem instance (factoring binary conditions) for an easy problem instance (checking binary conditions). Functional programming has a weird dual personality where on the one hand you hear "A functional programmer is always a smarty and solve hard problems as a matter of course" but also you hear "functional programming would be the dominant paradigm if only ... we taught people young so they wouldn't have their bad procedural habits"
Imagine a case where you have 4 options. W, X, Y, Z.
Y and Z are mutually exclusive.
X can only be set if W is set.
If Y is set then X must be set.
Going down this road you end up encoding your business logic into your datatypes. Which is good to a degree, but makes things messy when new options are added or requirements change. Imagine a new option U is introduced that is only valid when W is unset and Z is set but allows X to be set. Your interface becomes very hard to manage with even a small amount of complexity added to the options.
True. And what is an "illegal state" anyway? If two options are "mutually incompatible" it means that they, as far as we know, so far, have never occured at the same time.
But that is just how things are currently. The world may change. And it's important to strive for maintainable code that can accommodate such change easily.
Expanding company operations to work in a different country is an example of this (our) "world changing". States that never occur together here, may occur together there. Or in the future more generally.
So, making illegal states non-representable is about avoiding errors in respect to the system specification. But it does not take into account the fact that in the real world specifications typically evolve and change even during development, and perhaps more so later.
This is technically correct but disingenuous. This reminds of the climate change comic where a scientist asks “What if climate change is a big hoax and we create a better world god nothing?”
This is an excellent illustration. I feel functional programming provides great tools and concept for a cohesive system (one with consistent requirements, which accomplishes a single thing, etc). But many programs you write involve parts that aren't very cohesive and trying to make them cohesive is far more work than just bolting on a case statement or similar things.
Parent currently downvoted without comments. Seems like a sad way to respond.
I feel like you might be missing the point. A case statement is not something you bolt on as a hack. It is the right tool for the job the vast majority of the times. When you use case statements, you refine and reduce your state space. It makes code easier to understand. When you combine this with the idea of making illegal states unrepresentable, a case statement gives you an exhaustive listing of what could happen. Even a lot of Haskell programmers, after using things like the `maybe` function to eliminate Maybe types and things like `fmap` over Maybe, eventually find that using case expressions produces the clearest code.
I really hope HN enforces a rule that a downvote must be accompanied by a reply.
If you only have 3 states, then yes, that should be an enum, not a pair of booleans, because you have 3 states, not 2x2 independent ones. Making the 4th state unrepresentable removes code and error checking. It's also just simple domain modeling.
Your latter example needs context. In what situation have you had an enum with 2^16 states? In any case, if you generally have a reasonable number of booleans, with some states being logically invalid, then you'll need error checking on those anyway.
Leaving them as booleans gives you the ability to monkey-patch things later in an ad-hoc manner, which is useful when you're still in a prototyping phase and the requirements aren't clear (and you could argue that this is almost always the case in many problem domains; I think that would be valid criticism.) But if you've already modeled the domain and you want any assurance of correctness, then I don't see why you wouldn't enforce the constraints at the type level. Your criticism seems to me the usual trade-off between strong static typing and dynamic and/or weak monkey-typing.
Where does this exponentional size requirement come from?
Imagine you have a program where there are 4 options, W, X, Y, Z. Y and Z can't be set at the same time, and X can't be set unless W is set. If Y is set then X must be set as well.
How do you represent this in a way that makes it impossible, even through programmer error elsewhere in the program, to have the flags in an invalid state?
You can create en enum that looks like:
Maybe I'm spoiled by TypeScript but this is how I'd do it in a structural typing system:
And the Make Impossible States Unrepresentable crowd program like that?
FP nerd: The pure core is nice and composable, with the imperative shell at the boundary.
State Skeptic: Yes. But! How do you compose the 'pure core + impure shell' pieces?
FPN: Obviously, you compose the pure pieces separately. Your app can be built using libraries built from libraries.... And, then build the imperative shell separately.
My take is that the above solution is not so easy. (atleast to me!) (and not easy for both FP and non-FP systems).
Take an example like GUI components. Ideally, you should be able to compose several components into a single component (culminating in the app) and not have a custom implementation of a giant state store which is kept in something like Redux and you define the views and modifiers using this store.
Say, you have a bunch of UI components each given as a view computed as a function from a value and possible UI events which can either modify the value, remain unhandled or configurable as either. Ex: dialog box which handles text events but leaves the 'OK' submission to the container.
There are atleast two different kinds of composability (cue quote in SICP Ch1 by Locke) - aggregation and abstraction. Ex: Having a sequence of text inputs in the document(aggregation) and then abstracting to a list of distances between cities. This abstraction can puts constraints on values of the parts, both individually and across parts. There is also extension/enrichment, the dual of abstraction.
This larger abstracted component itself is now a view dependent on a value and more abstract events. But, composing recursively leads to state being held in multiple layers and computations repeated across layers. This is somewhat ameliorated by sharing of immutable parts and react like reconciliation. But, you have to express your top->down functions incrementally which is not trivial.
> But, composing recursively leads to state being held in multiple layers and computations repeated across layers.
True, which is why re-frame has a dependency graph and subscriptions that avoid re-computation, i.e. the data dependencies are outside any view tree.
If data changes, only active nodes (ones that have been subscribed to) will re-compute. If nothing changed in a node, any dependent nodes will not re-compute.
It's a beauty.
FP is not a silver bullet. GUI is the classic OOP showcase.
> Ideally, you should be able compose them several of them into a single app and not have a custom implementation of a giant state
If you are suggesting that components store their state, I'm not sure about "ideal" there. That works well for small GUI applications. In GUI applications of modest size, you do want a separate, well-organized and non-redundant data layer you can make sense of, at least from my experience. Qt, specifically, allows you to do both things.
To your main point, I wouldn't say exactly that the component stores the state. But, rather that every component provides an initial value, possible events, and a default event handler which is a function from value to value. In effect, this is partially 'storing local state', but the above pieces can be composed to create a container component.
Note that there is no option really - the app wont be reimplementing how a key is handled in a text box. But composability means that the same principle should hold not just for OS/browser components but also for higher level components (A custom map or a tree-view where there are restrictions on types and number of nodes - these should also have default handling and delegation to upper levels.)
The global store choice makes it harder to have component libraries. But, the composable alternative has its problems too - redundancy and communication which skips layers (which requires 'context' in React).
This is a digression, but regarding OOP, my somewhat provocative view, is that it is not a natural thing, but in most languages, it is atleast 4 different concepts 1. Encapsulation/Namespace, 2. Polymorphism, 3. Extensibility(Inheritance is a special case) 4.Mutability.
These four concepts are forced/complected into a 'class' construct, but they need not be.
In particular, FP only varies on 4, but languages like ML,Clojure do 1,2,3 even better than OOP languages. Modules for encapsulation, Dispatch on first or even all arguments for polymorphism and first class modules, ML style, for extensibility.
Aside: There was a recent post (https://osa1.net/posts/2024-10-09-oop-good.html) (by someone who worked on GHC no less), favorably comparing how OOP does extensibility to Haskell typeclasses, which are not first class, but modules in ML languages can do what he wants and in a much more flexible way than inheritance!
There is also the dynamic aspect of orginal OOP - message passing instead of method invocation, but this is about dynamic vs static rather than OOP vs FP.
What OOP languages have managed to do which static FP hasn't done yet is the amazing live inspectable environments which lead to iterable development like we see in Smalltalk. The challenge is to do this in a more FP way while being modular.
Interesting link, thanks.
This page (https://reasonml.github.io/docs/en/module) is useful to see how a FP language can do what he wants. Because we have functors, which are functions from a group of modules/classes to another module/class, we can have Composition, Inheritance(single/multiple), Mixins etc.
Completely agree with these.
One way to achieve "Make illegal states unrepresentable" is by using "refined" types, a.k.a. highly constrained types. There is a "refined" library in both Haskell and Scala and the "iron" library for Scala 3.
I'm very disappointed. I was really hoping for something like the SRE affirmations - https://youtu.be/ia8Q51ouA_s
It's hard not to giggle when the conclusion right after "Smart constructors" says "Do these ideas belong only in functional programming? While they are practiced more there...".
Ah yes, because using constructors to ensure that new objects are in a valid state is virtually unheard of in object-oriented programming.
One thing this article does is assume extreme functional mindset, I dont even think OOP enters into the authors mind- With that context, I think that statement isn't about object constructors but type constructors.
A class is just a function in the category of Javascript.
This is fine but it's just a rehash of old well-knowned stuff.
I don't see the value of learning this stuff one random blog post at a time.
There are many books and established blogs with long series of material to give you an education.
> This is fine but it's just a rehash of old well-knowned stuff.
A significant share of the current dev community wasn't there when these principles were first described. Sometimes, bringing up old topics once again helps seeing that many people had never heard of them yet.
Known by you, maybe.
None of these "affirmations" are common in any code base I work with, unfortunately.
Do you have an example of the many books and established blogs that you can share? A newcomer might not be familiar with where to start when attempting to learn more about functional programming.
I'm a big fan of Haskell Programming from First Principles. That's where more advanced ideas like Monads started clicking.
https://haskellbook.com/
Let's keep repeating it until it gets more adoption.
It's a pretty good collection and summary of ideas I've only seen scattered around, before.
Could you list the top 5 ideas, with a short summary and a link to a well-regarded blog post that goes into more detail for each?
~~I can't tell if you're being sarcastic, but that's exactly what TFA does. But just to pull it out explicitly:~~
Edit: I think I misunderstood your point. You were asking for a similar kind of list as TFA from the other commenter which may not necessarily be the *same* 5 ideas
Leaving the rest here in case it helps someone:
1. Parse, don’t validate
https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-va...
2. Make illegal states unrepresentable
https://fsharpforfunandprofit.com/posts/designing-with-types...
3. Errors as values
https://jessewarden.com/2021/04/errors-as-values.html
4. Functional core, imperative shell
https://www.javiercasas.com/articles/functional-programming-...
5. Smart constructor
https://wiki.haskell.org/index.php?title=Smart_constructors