Syntax Highighting - Testplan 2018-08-17

By Max Woerner Chase

In this post:

So, I started writing up test plans, had a few good ideas, and then I hit a snag. See, section 2.3 of the Python language reference is Identifiers and keywords, and in my opinion, the highlighter I have to work with handles them shamefully.

No, no, try to be positive.


I'm positive that—

Okay, calm down, try to present this rationally.

The sense I get from the page on Scope Naming is that some of the ideas for what to highlight, like the name of something that falls under excessively narrow criteria to be considered a function call, came from working with languages where different kinds of thing have their own namespace, and which kind/namespace applies depends on how the thing is used. But with languages like Python and Lua, there's nothing like that, there's just values with names, and you can do stuff to the values, such as use them as function calls. Or add two values together, surround them with parentheses, and call that.

I got a little carried away trying to reconcile the mental model encoded in the scope naming recommendations with the documented behavior of Python. The end result is that about three-quarters of the section on Reserved classes of identifiers is about how I think the syntax should work. The other quarter is also about how I think the syntax should work.

Before my little rant there, I got three good section writeups, three that were just variations on "This section is about whitespace, I probably don't want to call it out", and the section on Whitespace between tokens, which doesn't merit tests of its own, but does suggest other tests down the line.

My plan for turning these test plans into tests is to write the tests I'm confident the syntax has a chance with, which are Comments, Encoding declarations, and Explicit line joining. After that, instead of working with keywords and identifiers, I'll start writing up and testing literals, because, again, I'm more confident that I mostly agree with the syntax on those.

On the name front, I've been tossing out all sorts of names. Like "Python Forward". I don't know, I need something that properly conveys the core philosophy here, which, on reflection, is probably something like "Well, I like it".

Next time, I start writing tests.