Unblocking myself by writing compilers
Part of my typical flow working on Derw is to build things using it. I'm a believer that if you just write the language but don't use it, it's easy to become estranged from the regular user and their pain points. With that in mind, I've used Derw for several projects in areas I've thought it might best suited for - typically writing frontend applications. As I rewrite the Derw compiler into Derw itself, I found myself blocked on rewriting the parser.
I think there's several types of being blocked - you might be blocked because you're unsure what to do. Maybe the spec is unclear, or you've never explored the problem space. This is not what I was blocked on, since I'm deeply familiar with Derw, I have a number of projects of real code using Derw, and there are thousands of integration and unit tests to back up any changes I might make. Then you might be blocked by other code changes - again not a problem for Derw, since I'm the solo dev and tend to branch out if I have a particularly long living side-track. I tend to prefer to stick to one problem at a time. The blocker I was facing is the one that hits me most: I know what I want to do, I know how to do it, I know it won't break the main branch. But I know the way of doing it won't be fun. It will involve a lot of messy, hard to maintain code. It will require arguing with the compiler, and less than perfect syntax to represent the concept.
The parser as written is full of little battles with the TypeScript compiler. Little bits here and there that require TypeScript-style code that doesn't translate well to other languages. These types of tricks make it difficult for the Derw equivalent to walk the same path. It's possible, I know what needs to be done, but there comes that blocker: I just don't want to when it's this awkward.
When I hit a blocker generally, I take some time away from the problem. I mull it over in my head while I'm doing other things. There's a common pattern of things that help: talk to a duck, talk to a person, write a blog post, take a walk, have a shower, go to sleep. Eventually something in your brain just clicks and you'll be back on your feet. I've done most of that, in particular one that helps me most: taking public transport rides for a length of time. During a train ride, I decided to program myself out of the hole by programming more: exploring parsers written in Derw. Since they didn't already exist, I created some: html parsers, language parsers, config parsers. The goal with these projects wasn't to solely create useful or interesting tools, but also to see how writing them felt. I'm fairly up to date on parsing literature, but the way I've always learnt best has been to make things myself. I wouldn't say the Derw codebase is a good example of a parser - but it is an example of a parser that works.
After spending some lunch times on these projects, and Neuro-lingo, I settled on a concept I'm calling lunchtime languages. Small languages that can have their compiler written on a lunch break. The more I thought about it, the more I thought about BNF and PEG - languages which can represent the grammar of a language. These languages are great when you're at university and try to see all possible structures a program may have in a specific language. But what if there was a way to naturally represent a language, closer to how actual code in a language actually looks? A lot of the existing grammars do make an attempt to replicate a language in a recognizable way, but as part of the exercise of creating parsers, I wanted to make one that would enable my experiments with parsing along with the lunchtime languages. The goal is to have a PEG style language which I find easy enough to generate parsers from in the course of a lunch time.
So that's where I'm at. I'm writing parsers as often as I can. I'm spending my lunch times coming up with fun languages with quickly written compilers. I'm working on getting Derw enough features to make parsing fun. There's some things that Derw already has that make parsing fun - like multiple unpacking of arguments in pattern matching. But there's things like having else-ifs as part of the language, and a standard library for chaining utility types that would help. My train rides helped un-block me in my direction: now I will follow that direction until the parser is done.