Don't make me read your PR description

I started using Github Copilot a month or two ago for my personal projects. I can’t say that it’s been a net positive. There are more than a handful of problems that make me slower:

1. Sometimes it suggests a closing quote or paren that isn’t needed. Removing the duplicate character is a pain. Because it’s a syntax error, formatters like Prettier or yapf start failing. Copilot isn’t great about following code style, so if you keep typing, it can sometimes be tough to track down which character needs deleting.

2. Copilot sometimes suggests extremely plausible-but-incorrect method or property names. It’ll add utterances instead of utterance_data. Frustratingly, Pylance and mypy both get this right, but Copilot does not—even if there are examples of the correct property/method in the file.

3. Sometimes it makes a suggestion that’s so trivial that it just gets in the way. If I type requests.get(‘https://foo/bar/’, headers), my cursor is after the s in headers and the closing paren was added automatically by VS Code. Copilot often suggests ={, which is…not useful. I was already going to do that. It’s more thinking.

There are lots of little papercuts, but I won’t enumerate them because that’s not the point of this post. The way I’ve found to use Copilot to be net-positive on productivity is to give it direct, actionable comments that are very exhaustive. Even when I’m coding on a machine that doesn’t have Copilot enabled, I tend to keep writing those exhaustive comments, because I’ve found that they help me think. It’s a habit now.

One thing that stands out to me is how often I and my coworkers (at any of my jobs over the years) have relied on PR descriptions to communicate intent. That’s the worst place to put context about the code. And there are two good reasons:

1. It’s hard to get to the original PR.

2. The information in the PR is potentially made obsolete by any changes to the code since the PR was landed.

Getting to the original PR means looking through git blame. For old code (especially in a large repo), that could mean peeling back many layers of commits to find the commit that actually landed the bit of code you care about (versus moving it around, formatting it, etc.). Github has a handy “view blame prior to change” button that I’ve used countless times, but it’s still challenging to find that original commit.

Whoever at Github added this feature: thank you from the bottom of my heart.

But more importantly, in the same way that code is applied in diffs, assumptions and constraints are applied to the codebase in diffs. The PR description is a snapshot of information in time. It never changes and it’s immediately out of date. Understanding the context from the PR description isn’t possible without looking at all of the other PRs that affect that code in the future and understanding what the intent was.

Moreover, PRs (or any other related resources) get lost. Notably, Stripe moved (at some point, well before I joined) from Phabricator to Github. There are plenty (well, a handful) of places where phab reviews/tickets/etc is referenced, and that data is lost to the sands of time. Keep in mind: data not part of the git repo itself (that is, data that’s not pulled with git clone) is stored separately from the codebase! If you switch from Github to Gitlab, you’re liable to lose a lot of context.

Because I’ve been prodded to write really exhaustive comments about what my code is doing (or should be doing) by Copilot, I’ve realized that my PR descriptions are just a summary of my work. I’m not explaining anything: the code should be self-explanatory. My thought process should be there in the comments! If you need to look at the PR description, it means my change doesn’t explain what it does.

Moreover, when I change the code in the future, I should be changing the comments to reflect the new behavior of the code. Now, there’s a record of that change in git, and I can see how and why I changed the behavior. Future me (or future anyone else) doesn’t need to play digital archaeologist to figure out what the hell I doing when I made a particular change.

And on top of that, the details about your code live in the git repo itself. Even if you delete the code’s comments, you can still look back in git history to find the comments you deleted. Github issues won’t survive deletion, migrating away from Phab won’t preserve your reviews, but the code that’s alive and well on your Macbook will live on (so long as you still have a non-shallow clone, anyway).