From long shot to one-shot
There’s a part of Twitter that’s been on-fire lately, talking about AI. It feels very vibrant. People are building things, discussing the best models, the best setups, completely new tooling is emerging. There’s a lot of excitement. Some in-group lingo is emerging. Let me give a couple of examples:
“One-shotting”: getting from zero work done to having the whole thing you wanted to accomplish completed after a single prompt.
“Vibe-coding”: interfacing with AI as the means of creating and iterating on a programming project and never interacting with the source code directly.
And in the same way you can find a proof-of-concept game created in the 30 minutes that is now a multiplayer game or even funding for vibe-coders, there’s an equal or greater amount of people reporting that AI feels useless to them. A lot of them feel like they are getting gaslit. It all looks like marketing-geared demos with no applicability in the real world of professional software engineering.
At some point I became increasingly intrigued by the discrepancy in experiences and thought it was time to give it a go and start using AI. And I had enough success from the get go to keep me going and believe it’s a very useful tool. So if you’re looking to get started with AI, I have two pieces of advice.
Advice #1: Treat it as a new skill
It took you some time to learn to write code the way you do right now. You’re probably still honing your craft. And you know your way to do things is far from the only way. You’ve met other practitioners that have ways of working so different to yours that it’s almost a miracle that you’re able to collaborate. It’s a bit of a similar problem when it comes to AI. Your old dog needs to learn new tricks. Poke it with curiosity about what will happen.
Advice #2: Set yourself up for being rewarded
There’s a lot of demos of apps being created from scratch with AI. And for good reason. AI is easier to use in new projects or small projects than otherwise. Start small. You can, for instance, start a new library in a programming language you’re not an expert in. Or you can take something as small as a single simple function alongside unit tests and ask chatgpt to write a similar one. Increase the complexity as your results get better. Play with how you’re asking the questions and the existing code that you’re feeding it. Build up an intuition as you see increasing success. And ride that joy.
As I’m not aspiring to be an inspirational guru, let’s get practical. I attempted to one-shot a syntax highlighting plugin for VS Code for my programming language, called tenecs. I’ve never built a VS Code plugin. I’m not even an avid VS Code user. I’ll report it in the present tense, as it is a transcription of my notes as I went through the process. You can skip to the end to see where I ended up if you don’t want to tag along on the ride.
Let’s start by feeding it a sample file of a parser combinators library I started building (gist) as I think it has a good coverage over the language syntax. I’ll use the prompt: “This is a file of a new programming language I made, called tenecs. Please create a vscode syntax highlighting plugin.”. It suggested some files that I need to create and some steps to follow to create the plugin project. Looking at the files I can see some issues:
Wrong file extension. It should be “.10x” rather than “.tenecs”.
Support for single-quote syntax, that the language doesn’t have.
The pattern for number literals does not include the potential minus at the start, used for negative numbers.
The list of keywords contains a bunch of keywords the language doesn’t have.
Clearly a failed first attempt. I do happen to have a grammar definition I can give as context. Let’s try giving it just the grammar and the prompt “This is the grammar of a new programming language I made, called tenecs. The file extension is “.10x”. Please create a vscode syntax highlighting plugin.” Looking at the output of this prompt I don’t quickly find any problems except that it missed the syntax for block comments. It is only supporting single-line comments. This made me realise that I don’t have the definitions for comments on the grammar I provided. So it speculated correctly, to some extent. There’s probably other problems there I haven’t spotted yet.
For my next attempt I’ll provide the grammar again but the prompt is “This is the grammar of a new programming language I made, called tenecs. Besides that grammar, there is syntax for line comments and block comments. The file extension is “.10x”. Please create a vscode syntax highlighting plugin.”. I don’t even state what the syntax is, as it is the most commonly used by other languages.
Now that I couldn’t find issues with the generated files (but they probably exist), when I tried to follow the suggested steps to create the plugin I’m starting to run into problems. Apparently it didn’t generate an entry-point for the vscode plugin and therefore the recommended tool to package the plugin fails. So my next attempt will be the same prompt, but followed by “Don’t forget to create the index.js file.“.
It went in a completely different direction. Bunch of problems:
There’s no “index.js” file.
It’s missing some language constructs it had correctly done before.
It suggests very different steps and those don’t include the same plugin packaging tool.
I’ve clearly hit a barrier here of what I can easily get it to do. Time to switch the approach. Instead of asking it to make the whole plugin for me, I’ll follow the official Syntax Highlighting Guide, because after that all I really need is the content for two files that describe a language and those it has already generated. I’ll repeat the previous prompt and attempt to just use those two files.
And we have a working plugin! I’m using a theme I wouldn’t normally use to code but that seems nice to quickly spot syntax highlighting issues. The syntax highlighting isn’t quite right, though:
A declaration should have “:” and “=” as the same color.
“~>” is part of a type definition.
Let’s try to fix those two things with a new prompt: “This is the grammar of a new programming language I made, called tenecs. Besides that grammar, there is syntax for line comments and block comments. Please create a vscode syntax highlighting plugin. Make sure the "<", ">", "|" and "~>" are part of the type but all other symbols are bundled together.”
It really seems to have focused on bundling all symbols together and things are overall a bit worse. Perhaps if we remove that part of the prompt and go with “This is the grammar of a new programming language I made, called tenecs. Besides that grammar, there is syntax for line comments and block comments. Please create a vscode syntax highlighting plugin. Make sure the "<", ">", "|" and "~>" are part of the type”.
And it’s still far from the goal. Perhaps listing the symbols is not the way to go. Let’s try “This is the grammar of a new programming language I made, called tenecs. Besides that grammar, there is syntax for line comments and block comments. Please create a vscode syntax highlighting plugin. Make sure the function types and sum types share the same highlight”
Now it’s gotten close. But “:=”, “:?” and “=>” definitely shouldn’t be be half-highlighted. Let’s go with “This is the grammar of a new programming language I made, called tenecs. Besides that grammar, there is syntax for line comments and block comments. Please create a vscode syntax highlighting plugin. Make sure the function types and sum types share the same highlight. Make sure => and :? and := are highlighted.”
Now we fixed that problem but “~>” is no longer highlighted. Let’s go with “This is the grammar of a new programming language I made, called tenecs. Besides that grammar, there is syntax for line comments and block comments. Please create a vscode syntax highlighting plugin. Make sure | and ~> and => and :? and := are highlighted.”
I’m going to consider this one a success. It’s a working plugin. It has questionable taste in the details but that’s what the human touch is for. The last sentence of my prompt where I specify some symbols is for sure not something I would do on a first prompt right now, but that goes to show that either the work needs to be iterative and/or there’s a lot of intuition for me to build up.