Three Dollar Quill

Weekly Roundup 2026-04-14

2026-04-14T04:00:00-04:00

Wednesday: I started thinking in earnest about some hobby projects I want to pick back up.
Thursday: I toyed around with both projects, and laid some groundwork.
Friday: I tried some things for the calendar stuff which didn't work, so I replaced them with something that did. On the sound synthesis side of things, deciding that I wasn't going to worry about streaming turned out to be just what I needed to work out a promising-looking (to me, at least) system for handling streaming.
Saturday: Mostly on the sound synthesis side of things, I started refining the implementation of some advanced features.
Sunday: I did a bit more work, but didn't feel like going into detail.
Monday: I focused mainly on the calendar stuff; I made some improvements and plans, and considered real-world (ish) use cases that would motivate the kind of features I'd like for using this code in a worldbuilding context.

Next week, maybe I'll try going back to one topic per day, if I'm going to be working on these two things, plus another third thing that I haven't been mentioning.

Coding 2026-04-13

2026-04-13T04:00:00-04:00

I didn't touch the synthesis code yet today, but I did work through the calendar code some more. It's now at the point where very basic calendars probably work, but before I move on with things, I really should try to get this stuff properly organized.

Let's see what I want in the near and far terms:

Divide things up into small modules.
Implement more calendars.
Implement some time handling.
Implement proposed calendars for other planets, figure out what breaks as a result.

Let's dive into that a little before I wrap up. All of the formulas I'm working from assume, roughly speaking, that days in different calendars move in lockstep. In particular, there's one day per day. This doesn't quite hold if we then switch to considering martian sols. My gut feeling is that martian calendars should require a sol-based equivalent to absolute day counts, and that conversions between martian and terrestrial calendars should be done based on moments, with some kind of conversion factor I don't want to think about. (In a sci-fi setting, this conversion factor can be elided with the assumption of FTL communication.) The exception is that it makes sense to talk about, given a martian month and day (or whatever), which martian year has the corresponding date falling within a given year of an earth calendar, or vice versa.

Working through this might inspire some changes to the current implementation; we'll see. For now, I want to get ready to go, and not think too much about any of this for the rest of tonight.

Good night.

Coding 2026-04-12

2026-04-12T04:00:00-04:00

I tweaked and enhanced the various bits of hobby code I'm working on, but nothing too exciting yet. Let's see what's coming up:

I need to finish running through the logic for the synthesis sequencing loop. Main question to address there is when I want to do the checks for indefinitely sustained sounds.
I've got a bunch of code to adapt from various prototypes to get the calendar code working.

For now, I'm tired and I want to take plenty of time winding down.

Good night.

Coding 2026-04-11

2026-04-11T04:00:00-04:00

All right, let's see what I've got. I've got some notes on a possible set of interfaces for working with sound sequencing, but there are some areas of it that are definitely iffy. I think the broad strokes are solid, and I've just confirmed that python-sounddevice appears to work for playback, at least on my machine.

...

Now then, I started filling in some of the stuff I have planned, but I need to fully nail down some of the data types. Basically, I need to have something describing a "Sound", which I know needs to have an onset, a sample function mapping npt.NDArray[np.float64] to npt.NDArray[np.float64], (I will later consider working with a smaller dtype.) and either an end time or a callback to determine when it should end.

There are various problems or complications with this approach, which I'm not going to consider right now.

...

Okay, I futzed around enough. I am done for tonight.

Good night.

Coding 2026-04-10

2026-04-10T04:00:00-04:00

All right, it's getting kind of late, so let's wrap this up quickly, since I don't want a repeat of last night. Instead of working on sequencing notes in space, I made some plans related to phase modulation. In all likelihood, getting it to work like a physical synthesizer will require some kind of compromise, and be a prime candidate for Numba-based acceleration. While I was thinking about all of this advanced stuff, I tossed together a sawtooth sample function. I'll try to work out the sequencing stuff later, but first, some notes on the calendar stuff.

I experimented with customizing __init__ in an attrs class hierarchy. It didn't really work out, so I just went with a classmethod. In order to test this, I tossed some junk implementations at the various abstract methods I defined. The first thing to try to get done with all of this is to get actual implementations in there.

Now then, let's see what I need to do back in audio synthesis land. From an interface perspective, I believe I want to be talking about start times and durations, in seconds. The prototype code just did a bunch of np.concatenate calls, which, eh. Let's ignore the question of streaming for now, since it's just going to confuse me. The samples for each individual sound have to start at zero. And stuff gets added, and on reflection, it might actually be easier to think of this in terms of constructing chunks, and yielding them out to something higher-level to process, so maybe I get streaming for free-ish. The interface would, I think, look something like pushing a combination of start time, duration, and sample function into a coroutine, and getting chunks of audio data out. I'll have to sketch out some diagrams or something tomorrow.

For now, um, it got super-late.

Good night.

Coding 2026-04-09

2026-04-09T04:00:00-04:00

Okay, let's see what I can plan out, with a little more time. On the audio synthesis side of things, I have a function to produce an unscaled and unshifted sine wave, which is intended to be passed an array of samples. These samples are interpreted as fractions of a period. I wish to have more sophisticated means of sampling and processing later, but first, let's see what's required to produce a 440 Hz tone for two seconds.

...

Okay, I've got a function to convert an interval of time into samples. Next, I need to look into writing the big array that this represents to a wav file. Looking at SciPy's options, I see that I need to actually make some kind of decision about lots of things. I think for now, I'll clamp the sampled values to the range from -1 to 1, and then convert to signed 16 bit integers for the output. I'm less sure about the proper data types to use for intermediate steps. I think for now, I'll go with "maybe too big, but almost certainly big enough".

...

All right, everything (eventually) worked the way I wanted and expected it to. Thinking about relative utility, I think next I'm going to work on starting notes at different times, then basic envelopes, then scales. At that point, I can work on as much additive synthesis as I feel like messing with.

For now, though, I think I'll ponder the other project I was thinking about, making calendar stuff extensible so that it can handle funky made-up calendars. For that, I'm right now thinking about the equivalent of Python date objects, which are (probably) going to involve a year, a year and a month, or a year, month, and day. The trick is, different calendars will assign different days to a given month (and different days to a given year), give a month a different number of days, or use a different set of months entirely. (And when you get into times, different years could involve days of different lengths; in exotic enough orbital configurations, this could, technically, happen within a single calendar.)

Anyway, if we ignore anything within or comparing days, then the equivalent of timedelta can be used with any calendar. And one way of encoding a datetime equivalent would be "days since some common epoch" combined with "means of representation". Because the "means of representation" has different fields with potentially different types, I conclude that it is a class, with an instance attribute of the days since epoch, and computed attributes of year, month, day, etc. Then, there are constructors for specifying the desired values of those computed attributes, and perhaps constructors for returning the date of a given holiday in a particular year.

All right, that makes significantly more sense than my prototypes ever did, so I'm feeling good about implementing this. Later. Because it's super late again. I just got really excited to be doing stuff with tangible results again.

But seriously, let's wrap things up.

Good night.

Coding 2026-04-08

2026-04-08T04:00:00-04:00

All right, I said I was going to try to do something, and here's something. I took longer than I would have liked with some paperwork tonight. So I'm just going to lay out some plans.

I've been thinking more about audio synthesis and music, and calendar systems. Really getting into sinusoids, I guess. It's getting late, so I'm going to take some quick notes for later:

Many instruments can be approximated by a combination of pure harmonics.
Pure harmonics can be obtained via additive synthesis, or via phase modulation.
Phase modulation is of interest because that's how many iconic synth sounds are produced.
Additive synthesis can be used to produce negative harmonics, and other inharmonic sounds.
Obviously inharmonic sounds are important for instruments like bells and struck bars or plates.
Subtle inharmonicity can be used to change the consonances and dissonances of intervals in a given tuning system.
Either approach should be not-too-hard to implement via linear interpolation of a wavetable of a sine function, or simply calling numpy.sin directly.
Another aspect of an instrument is its envelope; I tried implementing ADSR before, and it wasn't clear whether it was actually doing anything.
I'm planning to work with numpy arrays, and initially just write the results to a wav file, but also look into streaming.
Would like to work with notes in a somewhat MIDI-like way.
Overlapping them, I've thought of using the heapq module to sort upcoming notes by onset time, and playing notes by time to finish contributing; this would potentially be a workable basis for real-time-ish effects.
Possibly notes could be separated into tracks, which could then apply pink noise offsets to timing and amplitude.
I want to start with writing a public-domain song to a file, using sine waves with no envelope.
Near-term improvements include envelopes, additive synthesis (including naturalistic inharmonicity) and phase modulation, streaming, and pink noise.
Further-term improvements include alternative scales, and inharmonics to match alternative scales.
I'll keep Numba in mind for this, since it's served me so well in the context of image processing, but I want to hold off on writing things for it right off the bat; I'd rather rewrite for Numba if I see that things are too slow.

Anyway, it's super late, so I'm just going to wrap things up now.

Good night.

Weekly Roundup 2026-04-07

2026-04-07T04:00:00-04:00

Wednesday: I gave some thought to what exactly the scope of Villeins of Tomorrow should be.
Thursday: I made a little progress, but got tired.
Friday: Bad time at work, okay time at video games.
Saturday: Trying to hold on.
Sunday: Tried to recuperate.
Monday: Still tried to recuperate.

Next week, I'm going to... do something.

Diary 2026-04-06

2026-04-06T04:00:00-04:00

Okay, let's see what I'm even feeling like doing. Playing Artisan of Glimmith a completely normal amount, apparently.

Aside from that, I've just been kind of hanging around. I think I need to find stuff to do not on any of my various devices, but also I have a few things I want to set up on them. For now, though, I'd rather just get ready for bed.

Good night.

Diary 2026-04-05

2026-04-05T04:00:00-04:00

I kept on taking it easy. Did a few things that I probably won't talk about, but whatever. Still figuring out how I want to deal with everything that's happening, and I guess that means a lot of slow days.

Good night.

Diary 2026-04-04

2026-04-04T04:00:00-04:00

I poked a bit at Villeins of Tomorrow, but I've been having a rough week, as I think I mentioned before. Thus, I'm trying to take things easy right now. Nothing in mind to talk about.

(Except to mention that I was closer to the end of Tiling Forest than I thought. Definitely interested in the full game.)

Good night.

Diary 2026-04-03

2026-04-03T04:00:00-04:00

Well, today wasn't conducive to creative focus. Work showed me that one of the really unpleasant parts of man-made horrors beyond your comprehension, is when you start comprehending them.

I don't wish to elaborate on that, so, um, let's see... I played a bunch of Tiling Forest. If I'm remembering my completion percentage right, there's going to somehow be a lot more going on than I can imagine, which is going to be interesting to see how they manage that.

Aside from that, I can't think of anything else, so I'm going to wrap up now.

Good night.

Villeins of Tomorrow 2026-04-02

2026-04-02T04:00:00-04:00

Here's the plan: I've got some disaster ideas that work with science-fiction planets, so that's how I'm sketching out locations for now. While I've come up with a few ideas there, right now I'm just... really tired. It's sort of early, but I want to wind down now.

Good night.

Villeins of Tomorrow 2026-04-01

2026-04-01T04:00:00-04:00

Okay, I've read through the end of the current prerelease, so let's see what needs working out...

The big question is what the scale of the playset should be. Is this a big planet-hopping adventure, or a single location with more detailed breakdown?

While I think about that, let's focus on what we know. The starting inventory should be drawn from the Stars Above table, and I want to find some way to bring in other tables in a "planet of hats" or holodeck adventure kind of way, but that is not core to the premise, and I need to be ready to let go. I need six disasters, eighteen NPCs, eighteen locations, and twelve goals. Possibly these numbers can be tweaked, but I figure I'll hold to the numbers for The Clockwork City to start with.

So, I didn't want to type my draft of this into this blog post and go scrounging for it later, so I'm putting it together elsewhere. I've addressed some of the questions and lacks that I noticed, but I've only just started on the fifty-four entries I think I need. I'm again really not sure what the scale should be. Is each planet a location? Are there several locations to a planet? Are the locations all part of a single spaceship in which the players are encouraged to act out a slapstick parody of Alien?

... Maybe I commit to having many more locations, organized into "biomes". Now there's a thought. Maybe not a good one.

Anyway, I'm feeling kind of out of it, so I'm going to wrap things up and get some rest.

Good night.

Weekly Roundup 2026-03-31

2026-03-31T04:00:00-04:00

Wednesday: I made some progress prepping for Lancer solo RP, which I'm not sure is a thing the devs even expected to be possible, but whatever.
Thursday: A bit more Lancer work.
Friday: A little more...
Saturday: So, like, part of the motivation for the solo RP stuff was to try to deal with burnout, so... it turns out I'm kinda burnt out.
Sunday: I switched gears jarringly over to Villeins of Tomorrow, my space opera RPG concept in which various typical space opera heroes are, like, kind of fashy, so you have to do something about that. For this iteration, I'm not bothering to come up with a whole system, and instead trying to turn it into an Eat God playset.
Monday: I read most of Eat God, and came up with some questions I'm going to have to answer.

Next week, I'm going to try to finish that up, and start sketching out the shape of the playset(s).

Villeins of Tomorrow 2026-03-30

2026-03-30T04:00:00-04:00

Oh boy. I really wasn't up for much today, but I did read much of the current pre-release of Eat God, which has given me a few questions to ponder:

How should the Villeins of Tomorrow playset justify the workings of Catalysts?
Do the different Creeds fit with the milieu I'm going for?
Is there a principled way to allow players to enhance the in-universe "plausibility" of a Rebellious Art by rattling off some technobabble nonsense? Like I think it would really add to stuff like vanishing from a locked room, just because nobody was looking, if it's accompanied in-character with something like "an elementary application of phased quasi-tachyon disruptions". Ideally with extreme ambiguity about which nouns the adjectives are even modifying.

Anyway, I kind of don't want to look at my laptop any more tonight, so let's wrap things up.

Good night.

Villeins of Tomorrow 2026-03-29

2026-03-29T04:00:00-04:00

I'm back on this, and I've been thinking about a few issues I was having with the design:

What kind of tone should I aim for?
What sorts of things should the player characters be able to do?
What sorts of things should the player characters be doing over the course of a campaign?

I initially wanted to have a "serious" tone, with "realistic" portrayals of "the enemy factions think it's like that, but actually it's like this", and I didn't even have that clear of an idea for the other two questions.

After having a few months for this to bounce around in my head, I concluded that a more irreverent tone should fit my rhetorical goals just as well, and should make the setting easier to work with. I'm thinking The Hitchhiker's Guide to the Galaxy as a major source of inspiration here, and some further inspiration from something like Poor Superman.

And thinking about The Hitchhiker's Guide reminded me of a particular quote from it: "[...] and small furry creatures from Alpha Centauri were real small furry creatures from Alpha Centauri". Although I definitely want to be open to human characters, I thought it would make sense to focus on the role of puppetry in science fiction. And that gave me an idea that let me offload a lot of my questions onto a pre-existing game: instead of trying to design everything from scratch, I can try to put together a playset for Eat God. Here are the tweaks that I think are needed, offhand:

Need rules for playing as a human.
Need to have some way of representing the four character backgrounds I want.
Need alternative forms of names for players to choose.
Need to at least consider an alternative framing to "eating god"; on the other hand, space opera does often have lots of gods out there, so perhaps it's not really a stretch.

From there, I can then hopefully put together some generic or specific scenarios; I ideally want at least one focusing on each of the three branches of the Complex.

My plan is to properly read over everything, take notes as I go, and then start putting something together. For now, though, I need to get to bed.

Good night.

Diary 2026-03-28

2026-03-28T04:00:00-04:00

I'm ready to start writing stuff up, but in order to do that, I'm going to need to feel motivated to actually use my laptop for more time than it takes to write up these entries. I'm trying to figure out what I need to feel a little better and more motivated, and I just don't know. I need to have some time to rest, I guess, and also, crucially, to actually use that time in a restful manner.

Speaking of which, I should get to bed.

Good night.

Diary 2026-03-27

2026-03-27T04:00:00-04:00

I'm dealing with some nonsense right now, but I am gradually refining the parameters of the solo RP campaign. Before I jump into the briefing for the first mission, I'm refreshing my memory and filling in gaps in my knowledge about the setting.

I got distracted reading, and I should get to bed now.

Good night.

Diary 2026-03-26

2026-03-26T04:00:00-04:00

Working on the first mission for solo stuff. I've got a couple worldbuilding questions I want to check the book for (like "How much 'stealth in space' does Lancer have?"), and I want to refine the stakes a little (I have determined that my goal is to retrieve... something), but then I should be ready to go, at which point I just need the energy and focus to get started and keep going.

Anyway, with that said, I really shouldn't be working on any part of this any more tonight. It's late, I'm tired, really there's no reason to draw this entry out any further.

Good night.

Diary 2026-03-25

2026-03-25T04:00:00-04:00

Every time I estimate how many puzzles there must be in Artisan of Glimmith, I fall short. Anyway.

After dinking around in COMP/CON for a bit, I've got my character, and my mech, and everything's named. Big issue currently is that I followed my heart a little when it came to mech and pilot loadout, and either I need to roll things back a little, or my first mission is going to be in zero-G to justify the systems and gear I took. I guess my impulse is to try it out, and give myself a mulligan if it's just totally unworkable to keep track of combat in those conditions.

So, the thing to work on now is to take a look at the solo resource I'm trying to use this time; I have others to fall back on if it doesn't work out, but I'm curious whether the one I'm looking at now ("Invisible Game Master") represents a minor tweak to something like Mythic, or if it's going to be qualitatively different. That's not happening tonight, though. I've already gotten distracted and need to wrap things up half an hour ago.

Good night.

Weekly Roundup 2026-03-24

2026-03-24T04:00:00-04:00

Wednesday: I came up with some new plans for Impliciula.
Thursday: I realized that my plans broke 1ML. Again.
Friday: I decided to take a break from programming and mess with solo roleplay.
Saturday: I was sick, but then I was less sick, but I still didn't get as much done with solo roleplay as I would have liked.
Sunday: Vibeo ganes.
Monday: Movie.

Next week, I'm going to really try to get further with the solo RP. I mean it.

Diary 2026-03-23

2026-03-23T04:00:00-04:00

No end in sight with Artisan of Glimmith. I'm actually feeling maybe a little worried on behalf of the devs. Like, how long did they spend on this?

Anyway, between that and traveling and movies, I didn't get anything done on character creation, and now it's time for bed, so I guess this entry is another dud.

Got to get to bed.

Good night.

Diary 2026-03-22

2026-03-22T04:00:00-04:00

Okay, so. I meant to come up with a name and callsign for my RP character. Then I got distracted with video games and videos and such. If I tried to get something done now, it would be bad.

Best to just wrap up and take things easy.

Good night.

Diary 2026-03-21

2026-03-21T04:00:00-04:00

I seemed to be getting sick earlier today, but I think my condition is improving. I played around some with Lancer character building. I'm about ready to start putting the mech together, at which point I will have to work on the question of just what is happening to start with. It is a little frustrating to be going so slowly, but I'm going to have to just focus on, I'm doing things, and I shouldn't have any specific expectation of what I accomplish.

For now, I'm going to wind down and play video games.

Good night.

Diary 2026-03-20

2026-03-20T04:00:00-04:00

I'm trying to shift gears away from Impliciula for a bit, because I'm not feeling great. I'm looking into trying out solo roleplaying again. I'm right now seeing just how far I can get with Lancer before getting totally overwhelmed. I'm assuming not far, but maybe I'll surprise myself. I've just gotten started, but I'm done for tonight. Tomorrow, I'll need to figure out how I want to record things and take notes.

But yeah, for now, I should get to bed and wind down.

Good night.

Coding 2026-03-19

2026-03-19T04:00:00-04:00

Okay, now that I'm figuring out how 1ML actually works, I'm seeing how I messed up in my attempt to bidirectionalize it. I'm fairly sure that the way I was thinking of implementing function application would have done it in a way not visible to the type system, which defeats the point.

So, to try to make things work properly, I'm going to put together some requirements that my version of elaboration has to satisfy:

Synthesizing judgments should produce identical types and elaborations
Checking judgments should accept the same types, and "pure" judgments should be extended to impure judgments in the obvious way, given an abstracted type.
Because checking can't jump into the middle of the subclassing machinery, this has to be accomplished by destructuring and reworking the elaborations from checking terms.
Efun will require some minor improvements to handle abstraction.
Eapp will need a comprehensize overhaul to "fake" accessing the type substitutions.
Eseal might be fine, but need to check.
Ewrap cannot replicate its semantics unless it keeps on using the full sugar; unclear whether this is a problem.
Eunw should be fine if Eseal is fine.

Actually, let's see if I'm getting Ewrap right. To actually replicate the semantics, we'd need to accept abstracted value wrapper types, and the abstractions would allow us to interact with the wrapped expression as if it had a concrete type, by, um, hm. Quite aside from the fact that my current draft of the elaboration has some trivial issues I can fix up, for the rest of it, I really need to consider just how load-bearing the precise semantics are, because I've thoroughly confused myself at this point.

...

Given that I'm already carving out an exception for if expressions to match what refinement types want, I don't feel bad saying that Ewrap keeping the old semantics is on the table, even if it does make the bidirectionalization in that context a little pointless.

So, where things stand now is, I could work on cleaning this all up, but I feel like I really need to take a break for a few days. Not sure how I'll accomplish that, but I feel it's what I need to do.

First thing I can do in service of that is finishing up this entry on a reasonable time frame. No reason to draw it out further.

Good night.

Coding 2026-03-18

2026-03-18T04:00:00-04:00

All right, let's take stock. I'm improving my understanding of 1ML, and I'm getting a general handle on refinement types, and that's making me think that it might make sense to work on refinement types before algebraic effects. This is because algebraic effects (or at least the implementation I'm looking at) want sum types to exist, but don't really have strong constraints on the design, while refinement types have a bunch of things to say about sum types, some things that I don't yet fully understand...

I've got some things that I'm pondering about refinement types in general. It's talking about stuff like the type of an ordered list, and I'm wondering if it's possible to express "is an ordered list" as a predicate using the grammar specified in this tutorial, because I think I would prefer "refinement of generic type" to "bespoke type that happens to have the same representation".

Anyway, some stuff that I've been thinking about but haven't written down. Of the expression elaboration rules in 1ML, the big ones to make sure I haven't broken them in my attempts to bidirectionalize are Eapp, Eseal, Ewrap, and Eunw. In each case, there is a variable (two, in the case of Eapp) that I'm replacing with an expression. This removes the implicit binding that was putting the rest of the program or module inside the existential. At a glance, the effects of this look mostly safe, though I need to take another look at Eseal. But Eapp...

Existential types on the "function" type in Eapp shouldn't be a big deal, but the problems come in when I tried to replace explicit subtyping with a checking judgment. It looks to me like one of the consequences of this is that abstract types on the "argument" get bound "inside" the function call, rather than "before" it, which I don't think really matters, but it is a difference. The big difference that has me questioning things is that I guessed(?) that the way to handle universal types was to convert them to an abstraction attached to the parameter type; the argument is then checked against this new type, creating an expression of the new type, which encodes type information from the argument, in a form that should be accessible to the rest of the function application.

The main thing I'm now understanding is that, unlike how I was thinking of things, the structure of a function really does determine how it has to be called, so that (at least in the context of the explicit version of 1ML), generic functions need to be called with proper type arguments, so that the first argument to a polymorphic identity function has to be a type to specify which type it is the identity of. If I'm going to figure out a way to make this nicer, it's going to be solely within the realm of 1ML, since algebraic effects don't have any bearing on this, and in this context, the refinement types paper I'm reading explicitly does not specify how types should be inferred.

Anyway, I'm still not totally sure my ideas for function application are valid, and I'll be legitimately surprised if I've written the correct elaboration for them, but if this works, I can finish specifying things, and then either work on phase polymorphism, or investigating whether I want to beef up type inference in some manner.

For now, though, I'm sleepy and want to wind down, um, a few minutes ago.

Good night.

Weekly Roundup 2026-03-17

2026-03-17T04:00:00-04:00

Wednesday: I thought I'd found a problem with my implementation.
Thursday: I realized that the problem was in my mind, not in my implementation (which, to be fair, is mostly in my mind at this point).
Friday: I decided to take another shot at understanding refinement types.
Saturday: I concluded that I should stop looking at pure sealing for now, and started thinking about how best to get feedback on this stuff.
Sunday: Following some technical difficulties in the morning, I got back to getting things ready for feedback.
Monday: I continued drafting my writeups, and didn't do much worth writing up on the blog.

Next week, I'm going to keep on writing stuff up until I've got enough rough drafts to properly refine.

Coding 2026-03-16

2026-03-16T04:00:00-04:00

Okay, I've got more stuff drafted to go on the wiki. I'll try to get more drafted and entered into the wiki, and hopefully I'll be ready to take it public soon.

For now, I'm sort of taking things easy, and I'm not going to try too hard to have a full-sized entry.

Good night.

Coding 2026-03-15

2026-03-15T04:00:00-04:00

All right, I rescued my laptop's input system from my own weird mistakes, and now I can type without text fields freezing up.

What have I done with this? Frankly, not much. I've figured out which host I want to use for a wiki, and I'm drafting pages right now. I want to have something up soon, but right now is probably not a good idea.

What I'm working on right now is a high-level introduction to Impliciula, and later I hope to write up what I've actually accomplished and put it up in a somewhat digestible format. Right now, though, I want to get to bed.

Good night.

Coding 2026-03-14

2026-03-14T04:00:00-04:00

All right, let's see what I've been thinking of.

On reflection, pure sealing is probably too big of a risk to mess with, unless it proves to actually somehow be necessary. (This is because I'm planning to have sealing play a bigger role in "bidirectional 1ML", and making it pure would have adverse effects on the semantics of other nodes. I think.)

With that said, I'm trying to figure out what I want to do to have all of these concepts somewhere where people can look at them and discuss. The archives of this blog aren't great for that, for a few reasons. Ideally, I'd have a wiki that supports the same LaTeX packages I'm using for my current notes. That works for getting "definitive" writeups of the concepts involved, but I want to be messaging them out and discussing them somewhere. For messaging them out, my existing social media accounts should work. For discussion, wikis have discussion features, but they're a little weird. I'd at least like the discussion to be more public than "join our discord server", but I'm not up for moderating my own forum. Need to see what kinds of communities are out there that I can join and work with.

Anyway, I'm tired, and in a weird state of mind, so I'm going to go get ready for bed.

Good night.

Coding 2026-03-13

2026-03-13T04:00:00-04:00

I'm not really in a space for... accomplishing things, currently. Even so, I really do want to make some progress. I was just taking some notes on what I expect Impliciula to look like after I smush all of the features together.

Now that I have a general idea of how 1ML works, and a vague idea of how to integrate effects, I want to try and really sink my teeth into refinement types. Not happening now, and not sure if it's happening tomorrow, but soon. Right now, though, bed.

Good night.

Coding 2026-03-12

2026-03-12T04:00:00-04:00

Okay, the bad news is that I got distracted playing video games and I'm writing this up as fast as I can, late.

The good news is, I read over the various papers, and understand things well enough now, to be pretty sure that my laziness-motivated subtyping in System Fω shouldn't cause any serious issues. Basically, the generative/applicative distinction is driven by the behavior of binding an abstracted value.

Of course, now that I understand how purity interacts with functors, I want to take another crack at understanding pure sealing, although I really should prioritize effect polymorphism over pure sealing.

Anyway, it's late, better wrap up quick quick quick.

Good night.

Coding 2026-03-11

2026-03-11T04:00:00-04:00

I didn't think as hard about this as I might have liked, due to circumstances, but I believe that the semantics of my System Fω variant will break the generative/applicative distinction in 1ML. Fortunately, it's a simple fix in terms of changes to core code; the tricky bit is going to be rewriting the test helpers.

(The fix is "You know all of that fancy subtyping stuff? Don't do that.")

Before I touch anything, let's make sure I've got a handle on how all of this is supposed to work. Because now that I'm looking at this again, I think I might have thought about this wrong. What I need to do is, ignoring the semantics of my System Fω variant, figure out what 1ML "wants" to have happen with existential types.

I'm getting tired so I'm going to try to wrap things up, but my impression is that I should expect fundamentally incompatible return values from a pure function from a type to itself, and two invocations of an impure function from a type to itself. If this is right, then I just need to take some time to confirm that it works as expected/desired, and that it requires certain guarantees that my code doesn't currently provide. Anyway, tired, losing track of time.

Good night.

Weekly Roundup 2026-03-10

2026-03-10T04:00:00-04:00

Wednesday: I started to get a better handle on how things should work for semantic types and elaborations.
Thursday: I skipped ahead a little in planning, to try to avoid painting myself into a corner.
Friday: I got LaTeX basically working, although there were some... issues.
Saturday: I attempted to work out the implications of the changes I'm making to 1ML, and concluded that they're probably fine.
Sunday: Nevertheless, I did have to put a little more thought into things, because the elaborations are more annoying than the types (they kind of have to be, since working out the types without the elaborations is workable-ish, but vice-versa is either difficult or literally impossible).
Monday: I started thinking that the ideas I had for expressing purity notations really didn't make sense, and I should try doing it the paper's way for now.

Next week, maybe I keep with this, maybe I find something else to focus on.

Coding 2026-03-09

2026-03-09T04:00:00-04:00

All right, let's see. I've got rough drafts of all of the basic expression typing rules. Convincing myself that I've got convincing typing rules is a lot easier than getting the elaborations to look good. It's also been a pattern that the bidirectionalized typing can look more elegant by some standards, but the resulting elaboration is more of a mixed bag.

The big thing missing is a proper re-interpretation of the purity annotations. The more I think about it, the more the whole thunk-return instead of record thing just seems like stubbornness. I'll have to consider whether I keep on with that, or just do what the paper says. Of course, it is possible that the real answer is "just focus on the monad behavior, and the actual implementation is just a detail".

Regardless, I let it get late, and I have to go to bed. I'll maybe take a break from this for a few days; we'll see.

Good night.

Coding 2026-03-08

2026-03-08T05:00:00-04:00

I didn't get quite as far as I might have liked today. I copied over the binding elaborations unchanged, because I just don't think there's a good way to bidirectionalize them. Then I started on the expression elaborations, did a few, and then decided to skip ahead to rules for wrapped values so I don't forget them. In working on this, I found some potential issues that I think I addressed, but I'm frankly not really sure.

Basically, doing bidirectionalization means replacing a lot of X nodes in expressions with E nodes, because variables are synthesizing, and the syntactic sugar for using expressions instead of variables is also synthesizing, so if I want a term to check, it has to use expressions "natively". This sometimes results in different typing behavior from the syntactic sugar version, I think always involving existential types. For if nodes, I think there isn't a major difference, but I had to think a bit for function, application, and sealing nodes. Basically, bidirectionalized application needs more information in the function type, compared to the sugared version; this is because the argument type can carry more information. A similar story holds for sealing.

Now, unwrapping got weird for a similar but more involved reason. As before, the X is replaced with E, but then the type must be unwrapped, and used to generate a coercion function. Now, if we're working with an expression, then the type must handle being abstracted, which means that there could be a bunch of existential types outside the wrap type, and therefore the value could have a bunch of pack nodes, and by describing this, I've decided that my original implementation was probably wrong, but I've come up with something that I think has a colorable argument for being closer to right.

The current solution is, get all of the existential type variables, bind them, determine the coercion function between the wrapped type and the checked type, then generate an expression that unpacks every existential type variable from the wrapped expression, and only then applies the coercion function.

If this actually works, and I can keep on applying it, I should be able to get somewhere. Now, next thing to consider: hey, how should I implement union types and case expressions?

For now, I'm going to get to bed, since it's super late.

Good night.

Coding 2026-03-07

2026-03-07T05:00:00-05:00

All right, I'm back to thinking about bidirectionalization. Basically, there are several rules from 1ML that I want to bidirectionalize, and I'm not totally sure whether that's actually workable. The rules are:

Eif; I want to make it checking instead of containing an ascription
Efun; I want to make it checking
Eapp; I want to make it operate on expressions, and make the argument checking
Eseal; I want to make it operate on an expression, and make that expression checking
Ewrap; I want to make it operate on expressions, and make both the overall node and the argument checking
Eunw; I want to make it operate on expressions, but only the overall node can be made checking; the argument actually has to synthesize

And, like, I could just do this, but switching from sugar to "the node actually contains expressions" has consequences for typing that I'm not sure of the impact of. And these consequences only manifest in the context of "doing something with modules". Basically, in the context of the 1ML sugar, if an expression has existential type, then the pack statements get wrapped around the node's type, whereas with what I'm trying to make work, type checking doesn't send the packing back up, which I think means that stuff like sealing has to "know" about existential types which would simply be inferred under the pure synthesis paradigm. If I'm understanding this correctly, I'm then not sure about the actual consequences that this has for usability. Basically, how much of a pain is it to type functors in this paradigm?

Or, I guess, it's a matter of questions of usability, but also whether there are inferrable types that cannot be expressed explicitly? Well, always, the only way to introduce an existential type is through the type type. Now, I think for an expression to supertype a type involving an existential type, in the context of synthesis, the expression would have to reference the variable bound to the existential type itself. So, if we move the expression out of the existential, then it basically needs to just duplicate the relevant existential bindings. If I'm thinking about this right, it doesn't seem too onerous.

Well, that was some good thinking. I'll try to make more updates to my notes tomorrow, but right now I just want to take things easy.

Good night.

Coding 2026-03-06

2026-03-06T05:00:00-05:00

I asked around, and got some advice, and then, after subjectively far too much messing around, managed to get LaTeX producing broadly acceptable output. With this, and the advanced technique of "reading the paper", I've managed to produce my own version of all of the basic 1ML typing rules, including somewhat understanding where-types.

At this point, I'm ready to take notes on all of the changes I want to attempt, and then I can work out the proper file layout for implementing the basic level of syntax.

For now, though, I'm really out of it and want to take things easy.

Good night.

Coding 2026-03-05

2026-03-05T05:00:00-05:00

I didn't have as much focus as I would have liked, but I'm making a little progress thinking about what I'm going to do for semantic types.

The change I'm thinking about right now is extending semantic types to handle algebraic effects. It looks like I just need to say "okay, in the result type of functions, in addition to indicating the purity, there's going to be another function that maps an evidence vector to a 'control monad'". Now, the control monad is going to require an amazing amount of effort to parse, it looks like, but it does appear parseable.

Let's think a little about the refinement types stuff. I haven't properly read over much of that paper, but I see that it will probably take some changes; constructing the constraints involves running backwards through the construction of the environment. Not particularly challenging to support, but it's definitely going to be an implementation change. Furthermore, some modifications will need to be made to subtyping. All of these papers want to do different things to functions, and right now I can only hope that these things can be made compatible. Since 1ML does a bunch of stuff with existential types already, I could consider using them for synthesis in the context of applications, but that's probably not a good idea in the face of 1ML's small/large types distinction.

Anyway, it's late again. I'll see how much focus I can give this tomorrow.

Good night.

Coding 2026-03-04

2026-03-04T05:00:00-05:00

Okay, let's kind of start over with my ideas about semantic types.

For now, I divide them into pure values and computations, and impure values and computations. Pure values are the simplest representation of the type, and computation types are simply returning the corresponding value type. This means that base values are now directly the pure type, and abstracted or function values are thunks. For now, impure values are simply pure values wrapped in a thunk-return. We consider semantic types in general, separately from semantic types with purity annotations, even though general semantic types and pure values have the same representation.

The contexts where these different representations matter are that variables are bound with purity annotations, and the result type of functions is a computation with a purity annotation. Hopefully, it is then possible to ensure that all expressions infer or check annotated types.

Looking over the expressions to be synthesized, I now suspect the correct interpretation is that expressions are typed with annotated values types, but the synthesized expressions are computations corresponding to returning a value of that type.

This all seems reasonable. Past experience tells me that now is the time to step back and let this percolate for a bit. If it still seems sensible after that, I'll start filling in notes. Until then, I'm going to read up on refinement types to see if my ideas about how to incorporate them into purity annotations make sense.

For the immediate term, though, I'm going to try wrapping up this entry before it's suddenly midnight.

Good night.

Weekly Roundup 2026-03-03

2026-03-03T05:00:00-05:00

Wednesday: I tried to nail down the interaction between 1ML effects, and algebraic effects.
Thursday: I put together an attempt at a plan for getting this together.
Friday: I experimented with taking notes in LaTeX, unsuccessfully.
Saturday: I finally stopped misreading part of the 1ML paper.
Sunday: I tried to sketch out what my implementation of all of this is going to have to do, in concrete terms.
Monday: I took another crack at it, and challenged some of my assumptions.

Next week, I'm going to read up on refinement types for planning purposes, and have a long think about whether the way I'm thinking about "purity" makes any actual sense.

Coding 2026-03-02

2026-03-02T05:00:00-05:00

Okay, so. I guess what I realized while I was asleep is, I appear to have made a bad assumption about how semantic types work, which was making everything more complicated than it needed to be. Let's start over and see if I can get to a better place.

Where I believe I went wrong was in assuming that the elaboration rules were outputting complete programs. This led to a profusion of thunk-return wrapping, and more importantly, made it impossible to express certain function types in my fiddly System Fω variant. So, let's instead assume that they're outputting values, and the whole program is implicitly wrapped in a top-level Return.

This means that base types now just correspond directly to themselves, and many elaboration rules are now wrapped in a thunk. Anyway, let's try reworking the stuff I dashed out last night.

The pure identity is now of type FuncType(bool_, ReturnType(bool_)). The impure identity is now of type FuncType(bool_, ReturnType(ThunkType(ReturnType(bool_))))

The inner coercion functions f1 and f2 are now of identical type to the pure identity, and the outer coercions are now of type:

pure-pure: FuncType(ThunkType(FuncType(bool_, ReturnType(bool_))), ReturnType(ThunkType(FuncType(bool_, ReturnType(bool_)))))
pure-impure: FuncType(ThunkType(FuncType(bool_, ReturnType(bool_))), ReturnType(ThunkType(FuncType(bool_, ReturnType(ThunkType(ReturnType(bool_)))))))
impure-impure: FuncType(ThunkType(FuncType(bool_, ReturnType(ThunkType(ReturnType(bool_))))), ReturnType(ThunkType(FuncType(bool_, ReturnType(ThunkType(ReturnType(bool_)))))))

Let's try expressing this stuff in a code block, so I can have nice things like... indentation. Now, I've got to remember the rules. First, f1 applies to y. Then x applies to the result of that. Then f2 applies to the result of that. Then the purity coercion is applied. Maybe those last two go in the other order?

# General code:
Application(f1, y)  # ReturnType(bool_)
Sequence(Application(f1, y), y2, Application(Force(x), y2))  # Depends on purity of x

# Sorry to ramble in the middle of the code block, but this is really the crux of it all.
# The type of that sequence is going to be the same as the result type of the inner...
# I've been calling them identities, but they could be inverters or constants.
# Anyway, once we have either ReturnType(bool_) or ReturnType(ThunkType(ReturnType(bool_)))
# we need to apply f2, which, remember, is always of type FuncType(bool_, ReturnType(bool_))
# In the pure case, we can (probably) sequence the result directly through f2,
# giving us a ReturnType(bool_), which can be left alone,
# or sequenced through Abstraction(z, Return(Thunk(Return(z)))).
# (Or the whole thing could be wrapped in a Return(Thunk(...))?)
# Alternatively, we could be in the impure case, which means we have a
# ReturnType(ThunkType(ReturnType(bool_))), and we want to apply f2 in a way that gets us another
# ReturnType(ThunkType(ReturnType(bool_))).
# This will look like
# Sequence(x_res, v1, Sequence(Force(v1), v2, Application(f2, v2))) but with return-thunk wrapping... somewhere.

# pure-pure
Ascription(
    Abstraction(
        x,
        Return(Thunk(
            Abstraction(
                y,
                Sequence(
                    Sequence(
                        Application(f1, y),
                        y2,
                        Application(Force(x), y2),
                    ),
                    v1,
                    Application(f2, v1),
                ),
            )
        )),
    ),
    FuncType(
        ThunkType(FuncType(bool_, ReturnType(bool_))),
        ReturnType(ThunkType(
            FuncType(
                bool_,
                ReturnType(bool_),
            )
        )),
    )
)

# Looking over this, I believe the core of the purity coercion has to focus on the line
# Application(f2, v1)
# In the pure-impure case, this can be Return(Thunk(Application(f2, v1))).
# In the impure-impure case, this can be Sequence(Force(v1), v2, Return(Thunk(Application(f2, v2))))
# I think.

Okay, I've got a sketch of the concrete implementation (if we ignore type variables), but I think it would genuinely help at this point to think about monads. I now kind of see that the idea is, I think, to treat pure values as regular things, and impure values as a basic monad. To bring a value into the monad, it gets wrapped in a Thunk(Return(...))), and if f is a function of the appropriate type, then the binding operation on x looks like Abstraction(x, Abstraction(f, Sequence(Force(x), x2, Return(Thunk(Application(f, x2)))))), maybe.

What I'm getting now is that the particular monad implementation is, well, it's still important, but what I really need is a framework for working with arbitrary monads.

Now, when you get right down to it, the identity operator for proper types is technically a monad.

Anyway, I'm not sure if I'm looking at this stuff right. Like, for sure, pure-impure does a unit, and impure-impure does bind and unit. But ultimately, what is happening is that we have two monads, and three maps between them. So, given that, I want to step back from this again, and figure out how these monad types are supposed to interact when there's more going on.

My gut feeling is that, if I'm to get effects working at compile time, then the "purity" annotation has to go around everything else. (That's a load-bearing "if".)

Anyway, that's about enough for tonight. I'm going to wrap up early and ponder this stuff some more.

Good night.

Coding 2026-03-01

2026-03-01T05:00:00-05:00

All right, it's really late, but let's see if I can work something out. I'm right now looking into the coercion between the pure and impure functions from bool to bool. For various reasons, I believe the types of these functions in System Fω would be FuncType(ThunkType(ReturnType(bool_)), ReturnType(bool_)) and FuncType(ThunkType(ReturnType(bool_)), ReturnType(ThunkType(ReturnType(bool_)))), respectively.

Let's start by checking pure to pure, just to make sure we actually get a reasonable identity function.

Now, the coercion function in this case probably has type FuncType(ThunkType(FuncType(ThunkType(ReturnType(bool_)), ReturnType(bool_))), ReturnType(ThunkType(FuncType(ThunkType(ReturnType(bool_)), ReturnType(bool_))))), assuming I didn't get distracted while typing all that. The outermost layer of wrapping in the result type may be unnecessary, but I'm not currently sure how to evaluate that.

Anyway, to inhabit this, we need an Abstraction(x, Return(Thunk(Abstraction(y, ...)))) I'm not getting the inside of that right in my head. I'll just note that both of the internal coercion functions have type FuncType(ThunkType(ReturnType(bool_)), ReturnType(ThunkType(ReturnType(bool_))))

I'll also note that the outermost layer of wrapping does appear necessary, and it should go around the type abstractions, if any. Anyway, let's see if I can sketch out what has to happen here. The coercion function can apply directly to y, so then we have a return-thunk-return, which is sequenced into x, producing a return, which seems like it will cause problems.

I'm not going to look into it right now, but perhaps the coercion functions have to be pure identities rather than impure identities. That doesn't quite feel right, because then the function arguments all kind of run together. If the impure identity works, then we need to first thunk the entire computation up through sequencing into x, pass that to the coercion function, and then... sequence it into a force? That's either the obvious solution, or it won't work at all. I'll try to figure out which in the morning.

Either way, if this all works, then the pure to impure version would need to wrap the whole thing in a return-thunk, and the impure to impure... we'll see.

Anyway, I spent long enough on this, time to get to bed.

Good night.

Coding 2026-02-28

2026-02-28T05:00:00-05:00

We'll see about actually taking proper notes tomorrow, but I have at least finally figured out how to read some of the paper correctly. I might as well lay that out, since I don't know what else I would write this entry about.

Here's the deal: 1ML's elaboration is divided into several parts, and the part that I think will end up changing the least as a result of my syntax/semantics changes is the subtyping system. Now, subtyping works on what are called semantic types, which are a subset of System Fω types. (I'm right now planning to represent them using helper classes that can convert back and forth between actual types, for various practical reasons.) In the 1ML papers, subtyping relates two semantic types (and sometimes a sequence of "paths", which are a particular kind of semantic type), and produces two pieces of information that are required for the elaboration: a sequence of type variable substitutions, and a coercion function in System Fω. The substitutions should be safe to use as presented, and the coercion function requires some light modification.

Basically, because I'm messing with call-by-push-value, I need to have the coercion function wrap a ReturnType around the, uh, return type. Then, the nested calls the paper gives can be replaced with sequencing.

Now, the part I'd been missing was how the substitutions are created, because I'd been misreading one of the rules. All substitutions ultimately derive from the "Sforget" rule, which equates a very specific form of path with a "small" type. The substitution given in the rule involves a type operation, and it wasn't until today that I realized that the type operation was inside the subsitution, rather than the other way around. Minutes of reading could have saved me hours of confusion. Anyway, the meaning of it was "So, you have a type variable that takes some number of types as arguments, here represented by type variables specifically; to unify this whole expression with a small type, replace the outermost type variable with a type function that takes all of those other type variables as arguments, and returns the small type". This substitution acts on the supertype, to make it compatible with the subtype.

Looking over what remains, I'm still not quite sure how "Sstr" divides up the path sequences, and I'm going to need to do some worked examples of effect subtyping, because effect encoding is something I intend to do differently than the paper, and it's going to need some form of coercion that I'll have to do some worked examples of, even though there are only three possible things that can even happen. Hopefully I'll have enough pep to work on that tomorrow.

Anyway, it's late, although not extremely late, and I want to go lie down.

Good night.

Coding 2026-02-27

2026-02-27T05:00:00-05:00

I've got four pages from the various 1ML papers printed out, and I'm trying to turn them into notes. Trying to use LaTeX for this was confusing and unpleasant, so unless I find some other math typesetting system to use, I guess I just need to fully commit to doing this all on paper for now. I kind of didn't want to, because I was thinking of how nice it would be to have TeX markup ready to drop into the docstrings, but there's no point to that if it's going to look awful.

So, I guess I'll try to work on that tomorrow or the day after. Today, I played video games. And now, I go to bed.

Good night.

Coding 2026-02-26

2026-02-26T05:00:00-05:00

Okay, I let time get away from me again, but hopefully not as badly this time. Having reviewed the literature on extensions to 1ML, and implementing algebraic effects, I've reaffirmed that there's no good way to skip ahead in implementing things.

Here's my current short-term plan:

Rewrite the 1ML elaboration rules in a form that does more to permit/require bidirectionalization.
Specialize the semantic types to my weird finicky System Fω variant.
Implement it all.
Write tests based on all applicable code snippets.
Given what I've learned from this, plan out the next steps.

For now, I need to get to bed and hope I'm better rested tomorrow.

Good night.

Coding 2026-02-25

2026-02-25T05:00:00-05:00

Okay, let's see if I can take some notes quickly. I'm working from the algebraic effects technical report, as well as the effect polymorphism and extended writeups of 1ML.

Both of these three papers have a mapping from higher-level types to lower-level types. The mapping I will end up with will draw a lot from 1ML, but will draw from alebraic effects, in that effects beyond the basics will have additional effects on the return types of functions. Basically, I think I need to figure out how to combine the two monads that the different papers are talking about, which is annoying in general, but hopefully doing it in a sort of specific context like this will make things not too bad.

Before anything else (and with not much after it, because I spaced out and it's super late), let's see if I can figure out what the signature of a map function looks like.

map : (a : type) ->p (b : type) ->p (m : effect) ->p ((a) ->m b) -> (list a) ->m (list b) Or something like that.

Now, here's what I know very offhand:

1ML syntax manipulations should handle a lot of this even before I get algebraic effects in the mix, and they shouldn't make too much of a difference to the implementation.
I definitely want to make sure that the type and effect arguments can generally be inferred.
My big concern is how to handle "oh yeah, sometimes the function type is wrapped in existential types, sometimes it isn't" from 1ML. I don't see how to do that, unless actually I'm wrapping the entire function type in a bunch of alternatives.
Regardless, the play looks to be "how do I integrate 'wrap this type in a monad' with 1ML's weird bespoke polymorphism?" Part of that question is "1ML semantic types have a lot of pieces; at what level does the monad wrapping go?"

Anyway, I'll try to come up with an answer to those at a not-messed-up time of day, later, since it's way too late to do it now.

Good night.

Weekly Roundup 2026-02-24

2026-02-24T05:00:00-05:00

Wednesday: I made some initial attempts at a plan for updating the semantics of Impliciula.
Thursday: I may or may not have been overcomplicating things. Legitimately not sure yet.
Friday: Oof, these are hard to read back. I get that math terminology is weird for taking an everyday term and giving it a very specific technical meaning, but I think "give the exact same term different definitions in different contexts" is worse.
Saturday: I redid the planning, with a better understanding of what any of this stuff is actually supposed to do.
Sunday: I tried to figure out what's going to happen with effects, and didn't really make much progress.
Monday: I switched gears slightly to pre-planning for algebraic effects. My hope is that this will be a not-too-extreme extension to 1ML with effect polymorphism.

Next week, I'm going to try to balance figuring stuff out here with not burning myself out.

Coding 2026-02-23

2026-02-23T05:00:00-05:00

Okay, I'm over-engineering my efforts to tweak the 1ML elaboration, so let's put that down for a bit. Let's take a look at algebraic effects in a non-ML context.

I'm right now working from Appendix C of Generalized Evidence Passing for Effect Handlers. This presents an elaboration to some form of System F, and hey, I've implemented some form of System F! Probably not the same form, but I'm having some trouble parsing out the fine details, currently.

In any case, let's go through the rules and see how much I'll mess with them.

var synthesizes the type
abs is on the one hand straightforward (check the type, bind the variable, check the body against the result type and effect to elaborate out the body, presumably wrap the result in an ascription that contains the transformed function type, which requires the same junk that has me stuck on 1ML, transforming a function type to reflect the effect of the effect), but on the other hand, reveals some differences with my own "some form of System F". Specifically, the expression/value distinction here is between introduction forms and elimination forms, with a "value" expression to implicitly bridge between them. My gut reaction is to treat computations the way this appendix treats "expressiona", and basically have introduction computations unify their effect with the empty row.
val would correspond to the Return node under this system; possibly the other introductory computations could be wrapped in a thunk?
tabs... the type-level distinction will have to be collapsed in some way when it comes to integrating this with 1ML; I'm not going to think about this too hard, but I am going to think about it. Things of note: apparently labels are the equivalent of a large type in 1ML, and also this translation barely does anything, which I'm going to hope is a good sign.
app has some confusing bits, but the key thing to keep in mind is that the elaborated code has a bunch of monadic binds, which are going to require that I support enums/unions/whatever.
tapp looks innocent until the elaboration, which, let's see, monadic bind, monadic translation, probably fine.
perform looks like it needs to synthesize, and it needs op to synthesize, I think.
ops has an unmatched brace in the elaboration that I think goes somewhere near the beginning (or maybe it should just be deleted). My gut impression looking at this is that it should be checking.
handler might be synthesizing.
prompt needs to synthesize from h, then has enough information to check e and synthesize overall.
yieldb has to synthesize from f, and then it either checks overall and k, or it does all synthesis and some unification; let's see if the former can work.
under, I'm not sure what to think; it's capable of synthesizing almost everything, but that's a load-bearing "almost". No matter what it does, there's going to be some kind of unification.

No telling whether I'll end up making use of any of these notes. I'm going to try to step away, and then investigate whether the monadic translation from page 22 of the technical report can be made compatible with 1ML semantic types. Right now, I really need to stop looking at screens.

Good night.

Coding 2026-02-22

2026-02-22T05:00:00-05:00

Slight update to yesterday: "Wait, 1ML sealing is pure under many circumstances, so there's a chance I don't need to try to get it pure in every circumstance."

In any case, now I'm trying to go through the elaboration judgments and modify and bidirectionalize them, and one of the early-on judgments is for function types, which ended up getting me distracted thinking about how to handle effects. My hunch currently is that I want to have two total rows instead of just one; one for "pure" and one for "impure". Although, this could lead to some weird issues, so I'm not committed to it. Another possibility is simply that purity and other effects are simply tracked in the same data structure, but in separate fields. The key thing is the semantics of effects, which must somehow make various changes to a function type.

I've been struggling to understand this better and have something else to say, but it's late, I'm tired, and I don't think I'm going to get any further tonight.

Good night.

Coding 2026-02-21

2026-02-21T05:00:00-05:00

All right, I didn't take any notes or touch any code, but I've got a working hypothesis about how the stuff I've been thinking about fits together:

Ascription in the 1ML sense is irrelevant to bidirectional typing, but not a big deal to implement. Furthermore, transparent sealing is also irrelevant, and confusing to implement, and also I'm still a little confused what the distinction is between transparent sealing and ascription. Anyway—oh. Okay, that's not transparent sealing, that's pure opaque sealing. That might be worth looking into later. Regardless, I can get by for now with impure opaque sealing, and that should fulfill the role of ascription in bidirectional typing. That said, pure sealing does sound good to look into later, in case I run into issues with function purity. Hard to guess whether I'll run into problems.

Anyway, reading over stuff about refinement typing has given me some things to keep in mind for choosing nodes, so here's my plan:

Get a fresh set of notes on node syntax.
Note likely changes to support algebraic effects.
Review refinement typing.
Decide whether to start with impure sealing, or to jump straight to pure sealing.
Wait, dangit, if sealing is impure, and I'm doing bidirectional stuff, then all of the function definitions are going to be impure, probably.
That's bad.
Okay, anyway, if that reasoning holds, then I'm going to need to properly understand the principles of pure sealing. Reading about "Skolemization" always freaks me out a little, until I realize I somehow did it by accident.
Anyway, I'm going to need to understand the nature of the pure sealing rewrite, because literally every new node I come up with is going to need to support it.

Here's how things look now:

For each node, determine the types of the fields.
For each expression-typed field, determine the typing direction.
Determine the typing direction of all expressions.
Bidirectionalize and work out pure sealing, in some order.
Effect polymorphism.
Algebraic effects.
Refinement types.
Front-end syntax.

Anyway, I desperately need to get to bed, so I'll get back to this at some point in the next few days.

Good night.

Coding 2026-02-20

2026-02-20T05:00:00-05:00

I've come to the conclusion that I was probably misinterpreting something in the last post, but I'm not totally sure what. I'm seeing that it could be that some of the stuff I thought was true about sealing in 1ML was a result of the syntactic sugar, rather than an inherent quality of sealing versus ascription. Unfortunately, this means I'm back to not being sure what the difference between sealing and ascription is.

I'm reading over a bunch of stuff, but it's not making things clearer, and some of it looks really scary to implement.

Let's take one last stab at this before I get ready for bed...

The difference would seem to matter in the context of a structure containing a type variable, and another type derived from that variable. If ML ascription and bidirectional ascription are the same thing, and distinct from sealing (I'm starting to feel very skeptical of this), then we have... I got the context wrong. If we have a functor, let's say that takes a type, and returns a record type, which we use ascription to coerce to type type. Does such a coercion result in inferring impure type for the functor? I believe I understand why sealing results in an impure type in this context. Sort of. And the sugar version of ascription produces a pure function which is instantiated with the relevant types, and, hm. That is indeed a pure function that somehow "exposes" the argument.

What it comes down to, then, is the question of whether I need sealing for semantics, and ascription for bidirectional typing, or if a slightly modified version of sealing would handle both use cases. One thing that complicates the question is that they have the same behavior on concrete types, so asking questions about how to handle weird type constructs is required to develop a proper idea of what's going on.

Anyway, it's late and I'm probably confused about something. I'm going to go lie down.

Good night.

Coding 2026-02-19

2026-02-19T05:00:00-05:00

Okay, I took a closer look at the problems I was seeing with my modifications to 1ML, and found... more problems. The top-level goal is, I want bidirectional typing because, as I understand things, that will help support refinement types. My initial plan was to "just" bidirectionalize 1ML elaboration, but it turns out that before I do that, I need to render it bidirectionalizable.

Here's what I mean: 1ML defines most of its expression nodes using variables rather than expressions, and provides syntactic sugar to implement ascription, and versions of nodes with expressions instead of variables. This is problematic for bidirectionalization because it limits the opportunities for checking; my general goal was to replace explicit subtyping checks with checking-based elaboration.

Now, my original plan was to make all of the changes I thought I needed, but that turns out to be difficult, because expressions and variables kind of behave differently. Or rather, some expressions type "like variables", and some don't. See, one of the distinctions that 1ML draws between types is "abstracted" versus "concrete" (I'm not sure if the paper uses "concrete" in this sense; it's been... a while since I read the whole thing closely). Variables and a few other expressions have "concrete" types, while generic expressions may have "abstracted" types. The difference is that "abstracted" types associate some number of existentially quantified type variables to a "concrete" type. This means that, when replacing a variable with a generic expression, in an elaboration rule, this means that suddenly there are a bunch of extra type variables that need to be accounted for.

For an example of this, if I'm understanding things properly, the major difference in behavior between ascription and sealing is that ascription entirely replaces the existential quantifications in the wrapped value, while sealing combines all of the existential quantifiers from the value and the type.

This kind of thing happens in a few other contexts, and seems to mean that I want to find some way of having a term synthesize its existential quantifiers, but check its concrete type. Now, it does look like I also want to be checking full abstract types in other contexts, so it kind of looks like I want three methods instead of two, maybe.

So, in order to have bidirectional elaboration, I first need to define the purely synthesizing elaboration for the nodes that I want to have. Then, I can figure out how to reverse some of the arrows. I could do that right now, but I don't feel like it, so I'm going to wrap up and find something else to do before bed.

Good night.

Coding 2026-02-18

2026-02-18T05:00:00-05:00

Okay, so, I'm trying to work on notes because I know if I just type in more code now, it'll be chaos, but I'm having trouble getting things formatted nicely; like, am I going to have to just take pictures of handwritten notes? Either way, I'm trying to avoid any large leaps, so here's the deal:

To start with, the 1ML syntax from the paper.
Plus support for first-class modules.
Now, adapt this to bidirectional typing, with some variables replaced with expressions, and note where using expressions complicates the elaboration logic (Eapp and Eseal, principally).
Now, make the changes required to support effect polymorphism, but not defining arbitrary effects.
Add the nodes and updates needed for algebraic effects. Note that aesthetics/symmetry indicate that it may be meaningful to speak of "large" vs "small" effects. At this point, I want to figure out whether any more work is required to support modular implicits, or if it's possible to support them via effectful functors.
Add or update as needed to support refinement types. Getting basic functionality is hopefully sufficient to replace the concept of "unsafe", but I also want to know whether the type-directed aspects of elaboration can create specializations of functions that explicitly lack effects, based on some kind of annotation.

I think in order to make progress here, I need to devote more work on Impliciula fully to taking notes and planning, and switch to another project when I want to actually write code. (Perhaps I could get back to increasingly-delayed Conlang Year?)

Anyway, I need to wind down now, so I'll call things here and try to get something more satisfying done later.

Good night.

Weekly Roundup 2026-02-17

2026-02-17T05:00:00-05:00

Wednesday: I decided to start reorganizing modules.
Thursday: I had some aspirational ideas about eventually improving the documentation...
Friday: I got far enough into the refactor to ponder pulling in more libraries to make the ergonomics nicer.
Saturday: I finished the refactor, and then moved on to 1ML instead of cleaning up what I've got.
Sunday: I started thinking in earnest about semantic types, and came up with some ideas that seem interesting, but may or may not work.
Monday: Not fully done with semantic types, I started poking more vigorously at elaboration.

Next week, I'm going to try to lay out the progression of things to work on; I've been sketching out how to get algebraic effects working, but I really shouldn't try to put any of this into the code yet, since it's only more moving parts, and I've got plenty of things to make sure I have working already.

Coding 2026-02-16

2026-02-16T05:00:00-05:00

I'm getting started on the elaboration logic. One thing that I'm not sure about yet is how I'm going to do stuff like signal failure.

Let's run through how things could go wrong, and see what it would produce.

Suppose a have a value of unit type, and I attempt to coerce it to bool with a type ascription. Fundamentally, that's going to manifest as attempting a subtyping relationship between incompatible base types. The easy solution is for that to fail immediately. More involved would be for it to somehow embed information about the 1ML code into a coercion function that deliberately fails type checking.

Thinking about it rationally, I crowbarred in more advanced type-checking code after-the-fact in the IR layer, so there's no reason I can't focus this pass on ease of implementation, and then fix up the ergonomics after-the-fact, but I would like to have some idea of what interface I'm going to try to implement, at some point.

Anyway, it's late and I should wind down.

Good night.

Coding 2026-02-15

2026-02-15T05:00:00-05:00

All right, I've got a lot of work to do to get the 1ML layer up and running. So, I decided to just jump in. I'm trying to put together some kind of rough draft for the semantic types.

The main obstacle here is that I've only got so much of an idea of how to get subtyping working, which is going to be the bulk of the work. It looks like I want to have distinct implementations for abstracted and concrete signatures, where concrete signatures can take arbitrarily many path arguments, for... reasons. The main obstacles I'm seeing here is that I'm only pretty sure that the correct implementation of the identity function (for base and path subtyping) is with Force instead of Return, and I'm not quite sure how to implement the type operators(?) that remap variables in some contexts.

I'm tired, so I'm going to stop writing about this, although I am going to keep thinking about it.

Good night.

Coding 2026-02-14

2026-02-14T05:00:00-05:00

I went into a fugue state, and now the refactoring is done. I haven't decided yet whether to go for the lazy loading stuff, and I'd rather move forward with other tasks, before coming back to this and making the call one way or the other.

Still need to work on linting, but I'm not feeling up for it right now. In the name of getting something done, I'll work on the stubs for 1ML.

...

Okay, that's done, and I don't want to do anything else tonight. Better wrap up.

Good night.

Coding 2026-02-13

2026-02-13T05:00:00-05:00

I'm not done moving stuff around, but I did end up upgrading the helper methods, although I mostly didn't update the tests to use the new capabilities. (This is partially because it's not clear to me how much I'll actually use the helper methods; I see much more obvious utility from the wrapper classes I wrote to support them.) Then I cleaned up the mess I made of the imports by going "Well, as long as the tests pass."

What I'm realizing now is that, although I kind of need to actually define stuff in a bunch of small modules, unless I switch to one huge one, handling the imports like that is actually kind of obnoxious, so now I'm looking into the lazy-loader project for a way to create lower-in-the-tree "facade" modules to give me nicer imports, that coincidentally would also remove a bunch of import-time dependencies between modules. I believe I should try to get through the rearrangement before setting up lazy loading, because I want to make sure I pull everything into the facade modules. Also because I definitely want to rearrange things, but I'm still not totally sure I want to set up the facades.

In any case, I did all that, and now it's late. Better get ready for bed.

Good night.

Coding 2026-02-12

2026-02-12T05:00:00-05:00

I'm still reorganizing everything. Once I have everything in the right place, I'll work on improving the ergonomics of the helper methods, then bringing the documentation back up to some kind of standard. At that point, I'll be ready to prep the labels and variables to support 1ML elaboration, and I'll hopefully be able to start implementing 1ML nodes.

For now, I need to get to bed.

Good night.

Coding 2026-02-11

2026-02-11T05:00:00-05:00

I was running into some typing issues, but I've resolved them. I decided that I'm ready to start pulling modules into the package skeleton I put together. This promises to be a bit of a long process, because I don't just have to move the modules, I have to split some of them up, and of course I have to update the imports every time I move something.

Once I have this together (which will probably take a few days), then I'm ready to start prototyping the 1ML layer. Which will prompt me to finally rework labels and variables, but I want to work from a nicer layout than what I initially ended up with.

Anyway, I let it get really late while I was fixing type errors, so I need to wrap up now.

Good night.

Weekly Roundup 2026-02-10

2026-02-10T05:00:00-05:00

Wednesday: I pondered how to reconcile 1ML with the syntax I want to compile it to.
Thursday: I took some notes on areas I want to focus on.
Sunday: I came up with an idea, which I haven't yet tried, for translating 1ML expressions to call-by-push-value terms.
Monday: I took some notes on changes I want to make, and sketched out a skeleton of new modules to fit my existing code into.

Next week, I'm going to try to fill in that skeleton, and maybe have another crack at some Python-level typing issues I'm hitting. The point is, soon I'll be ready to prototype the 1ML translation.

Coding 2026-02-09

2026-02-09T05:00:00-05:00

There are still some ways in which I don't understand 1ML subtyping as well as I'd like, but I think it's at the point where I just have to try to implement it to have any idea of what's up. I've taken some notes on my proposed modifications to the syntax, and I'm not sure I can do more to figure out what I need to know, beyond just trying to write the code. My barrier to that is, I want to fix up the code organization before it gets any more convoluted and arbitrary.

Let's run through my thoughts and see what I can get to make sense.

I want to move a lot of the existing "node" modules into an ir package, and within that, I want to divide them by feature.
So, we have the abstract classes (and putting them in a module called abc gets confusing when I want to work with the stdlib abc module at the same time), which need access to the Failure feature, which therefore needs to be importable without importing the module that defines the failure classes.
Some of the base classes require instances of the other base classes to function properly, so it would make sense to split up the base class definitions, as well. And the convenience method definitions would make more sense as class members, so that's another module for abstract classes; not sure what to call all of these.
Due to the way 1ML and other extensions interact with variables and labels, those should live outside the ir module.

Writing all of that out inspired me to put together a skeleton of where I think everything should go, but before I move things into place, I want to get the helper classes a little better behaved. I have some ideas there, but it's too late at night to try them out now. I need to wrap up as soon as I can.

Good night.

Coding 2026-02-08

2026-02-08T05:00:00-05:00

I didn't have proper internet for about two days there, so I've been closely reading over the 1ML paper. My goal was to create a blueprint for a 1ML-like language that fits naturally on top of the somewhat idiosyncratic version of System Fω I've been building up. This means supporting bidirectional typing, which is straightforward in some respects, but getting it right will require a better understanding of 1ML subtyping than I currently have. It also means supporting call-by-push-value, with the restriction of type variables to ranging over value types. I think if I can get that restriction working in this context, then I have a good chance of carrying it forward into the planned semantics.

My ideas for how to resolve this may break something, so let's try to pin down exactly what I have in mind here.

The obvious difficulty in reconciling 1ML with call-by-push-value is that 1ML terms are in some ways obviously values, and in other ways obviously computations, sometimes containing both conclusions in a single node. Call-by-push-value should have something to do with "purity", but the encoding of purity in the context of elaborating 1ML, using record types with reserved labels, to me just further obscures what this should represent.

So, I intend to try replacing these record types with specific transformations in the context of call-by-push-value. The key thing that rests at the center of my approach is noting that 1ML programs must elaborate to complete call-by-push-value programs; in other words, computations, not values. At the same time, the semantic types that guide the elaboration are associated with variables in the environment, which strongly suggests that they should be value types. Furthermore, many semantic types involve computation types, so the obvious resolution is to do something with thunk types.

Now, what makes this interesting is to notice the parts of the elaboration that definitely correspond to computation purity. These represent some but not all of the contexts where the type "should" represent a computation instead of a value. There are two purity annotations (pure and impure), and two "obvious" ways to convert a value to a computation (return and force).

What got this to make sense to me was to consider the pure and impure function types, and what their usage should correspond to in call-by-push-value. In 1ML, a multi-parameter function corresponds to several chained pure functions, ultimately returning either an impure function, or a pure function returning a base value. In call-by-push-value, a multi-parameter function corresponds to functions reducing to functions, until it reduces to a return type. What this suggests to me is that "purity" corresponds to forcing a thunk, and "impurity" to returning, and that the representation of a base type is a thunk of a return of a base type.

This all feels like it will accomplish something, but there are various questions that I haven't addressed yet, such as:

What's the proper semantic type representation for nested universal types?
What about record types?
Should I be working with higher-level representations of semantic types, or could it work to manipulate System Fω types directly? One point of confusion here is looking at limitations of the representation, and distinguishing bugs from features. Like, I can't convert a type variable into a pure computation in the context of typing; but is that a problem, or something I shouldn't even be trying to do?

Anyway, rather than try to write more, I'm going to publish this early and take another shot at understanding subtyping.

Good night.

Coding 2026-02-05

2026-02-05T05:00:00-05:00

All right, let's see where I am:

Removed explicit type variable construction, replaced it all with callbacks.
The type annotations are out of control again; I need to figure out how to shorten them.
Still haven't had the focus to rework variables and labels to work with 1ML.
Missing several node types required to implement algebraic effects.
1ML appears to require boolean types and associated nodes.
Refinement typing needs integer types, at least to get started with.
My fingers are still crossed that modular implicits will basically just fall out of combining 1ML and algebraic effects.

My plan is to alternate between refining the current implementation, and planning out what extensions I need to add these other capabilities. I'm intending to implement each one separately, ideally as a higher-level language that produces well-typed trees of the current implementation. I'm going to do them separately because jumping the gun in combining them won't make things any easier, theoretically or practically, and I want as few confounding factors as possible when things inevitably go off the rails.

I'm a little sleepy right now, so I'm going to wrap things up, take care of some chores, and then try to refresh my memory on all of these different theoretical frameworks.

Good night.

Coding 2026-02-04

2026-02-04T05:00:00-05:00

I got distracted from improving the code, by the question "Hey, what obstacles are there to using this implementation as a substrate for all of the higher-level features I want?" Right now, I'm focusing on the module language, because that's directly based on System Fω, and my implementation of it is... eccentric. As I recall, refinement typing is going to somehow tie into bidirectional typing, and algebraic effects is somehow going to come from call by push-value, and I've got those, so the major question is whether the tweaks that those two inspired me to make to System Fω will be a problem or not.

Ultimately, I've got a mixture of concrete goals and open-ended questions. Concrete goals include:

Provide additional implementations of Variable, in order to represent higher-level identifiers.
Provide additional implementations of Label, in order to represent variables.

Open-ended questions include:

The core semantics I want differ from traditional ML. Does this present any problems for adapting 1ML? Is there any chance this makes the implementation easier?
My implementation of System Fω is sort of impredicative. Is this enough for F-ing Modules?
The 1ML paper combines elaboration and typing; what does an implementation of this look like, especially given that I'd ideally like to still be emitting constraints?

I want to get a better grasp on the open questions, before working on the concrete goals, which should be pretty simple. I'm going to get off my laptop and try to make some progress there.

Good night.

Weekly Roundup 2026-02-03

2026-02-03T05:00:00-05:00

Wednesday: I started to get a better understanding of all of this code that I wrote.
Thursday: I speculated about how birds would divide up their higher-dimensional color space.
Friday: I thought a lot about how to name a new utility function that I decided I needed.
Saturday: I broke everything, but in a "something stronger will rise from the ashes" kind of way.
Sunday: I got into an argument with the type checker.
Monday: I've got an outline of an explanation of the typing concepts.

Next week, I'm actually going to try to switch to looking into F-ing modules, to see whether the restrictions I'm currently placing on System F make it unsuitable as a substrate for the module system.

Coding 2026-02-02

2026-02-02T05:00:00-05:00

All right. We were on the road a bunch today, and I ended up not on my laptop much, but I did put together an outline of an explanation of what's going on with typing, which I can use to validate the implementation. That's going to have to wait a day or two, because I need to wind down right now.

Good night.

Coding 2026-02-01

2026-02-01T05:00:00-05:00

A bit out of it and distracted today, but I did fix the tests that I broke in very strange ways, and got the tests most of the way to type checking. The exceptions are... obnoxious.

Basically, it made sense to turn a bunch of these helper methods I added into generic decorators, and to provide decorator-specific overloads to deal with the extra arguments. However, these arguments generally have related types, and it turns out that, if I don't actually use all of the functionality in the decorator arguments that I use in the function, then the returned decorator gets specialized with types that are subtypes of what I want them to be.

...

Got it. Very strange fix, but if it works, it works.

Anyway, I was able to apply this fix (adding an extra type variable to a specific overload) to some of the methods, but not all of them, so before I move on, I want to figure out what's going wrong with the functions where it doesn't work, because that's annoying, and I don't want to get a repeat of this when I start writing existential tests.

...

Well, I just did what should work, and slapped an ignore on it to deal with it later. The next obvious change to make is to update the existential helper methods, and that's going to be... interesting.

Any, right now I have to get to sleep.

Good night.

Coding 2026-01-31

2026-01-31T05:00:00-05:00

Well, I used a not-very-good name, just to get things moving, and then I implemented a bunch of improvements, and then some tests started failing, so I reworked one little bit of the typing logic in universal and existential types, several times. Once that was done, I was able to write helper methods that introduce fresh type variables for universal types, which on the one hand allowed me to cut down on some test code, but involved a truly horrifying set of type annotations, and also some of the tests are no longer valid, and more will be broken by future updates. There's so much to do to pick up the pieces here:

Figure out how to salvage the tests that will be broken by upcoming changes.
Update the existential helper methods to make those changes possible.
Make some non-destructive changes to the universal helper methods.
Document the assumptions going into all of this, especially the typing code that I updated to hopefully match the subsumption code.
Update the test code to take full advantage of the updated helper methods.

Anyway, I have to get to bed now. It's so late.

Good night.

Coding 2026-01-30

2026-01-30T05:00:00-05:00

Okay. So.

I've got a sufficient plan for implementing subsumption checking, with just one tiny obstacle: I'm not sure what to name the function that handles this.

The requirements I'm working with are:

Failure nodes subsume and are subsumed by all corresponding nodes. (Aside: for the same reason that I have this requirement, coercion failure should not emit an instance constraint when coercing a failure node.)
Otherwise, nodes of different types have no subsumption relationship.
Finally, any given class of node needs to implement custom logic, which may recursively check for subsumption of specific fields.

The way I decided makes sense to implement this is to define a protocol with a method that takes an argument of type Self, and define a top-level function that performs the required instance checks, then delegates to the protocol method. The protocol method, meanwhile, is free to call back into the top-level function. This top-level function takes two arguments that both implement the protocol, and returns true if the second argument subsumes the first, and false otherwise.

There are some other details that will make implementing other improvements nicer, but fundamentally, the important thing right now is that I'm not sure what to name the function. I could just name it is_subsumed_by or something, but I'd like something that's a little "happier" appearing in prefix position, rather than infix.

Once I have something I'm okay with, I can get to work on the implementation, and knock it out pretty quickly.

For now, I need to get ready for bed.

Good night.

Diary 2026-01-29

2026-01-29T05:00:00-05:00

I believe I fixed some latent bugs and inconveniences with Impliciula, but the next thing to focus on requires a lot more thought to get right, or even tolerably wrong, so I'm going to try to shift focus a bit.

So, tetrachromatic vision. Looking over the sources I'm skimming suggests to me that I'm missing some important nuances, but I'm going to just forge ahead for now. In at least some birds, the discrimination between blue and green is weaker than the discrimination between red and yellow. This is similar to humans, and suggests to me that "blue and green" could be one of the later categories of color to be discriminated for birds, just like is the case for humans.

Thus, I imagine a progression from "light and dark" to "black, white, and other" where shades and tints are part of black and white, and blue-green is generally lumped into black, yellow to yellow-red into white, and "other" is a continuum between red and ultraviolet, with ultraviolet being "fuzzier". I want to say the yellows and oranges get broken out next, followed by separating red from ultraviolet, then finally separating blue-green from black. Next, dividing up the fuzzy area around ultraviolet into distinct colors, in the order of an intermediate red-ultraviolet, followed by yellow-ultraviolet, then blue-green-ultraviolet, then dividing up yellow-ultraviolet more finely.

This is all working off of vibes, and isn't taking some things into account, so I think I need to work out the divisions of hue space graphically, at some point.

Anyway, that's all I have focus for tonight.

Good night.

Coding 2026-01-28

2026-01-28T05:00:00-05:00

As I said yesterday, I decided to actually properly implement subsumption. This will allow me to do stuff like avoid cascading errors when a coercion fails, so the user can focus on whatever caused the coercion to fail.

Now, this isn't really a snap-my-fingers deal. What I'm very confident of is as follows:

The parameter type of a function type is contravariant.
Most other types are covariant, probably including universal and existential.
Some types are apparently invariant, but I'm a little confused there.
Kinds don't have variance, so call them invariant.

Now, subsumption, in terms of the behavior of my code, is only relevant when a synthesizing node is checked. In the default implementations I wrote, the synthesizing node synthesizes, and then we check that the passed-in value subsumes the synthesized value. Currently, both of these values could contain or be failing node, just according to the Python-level type annotations. Let's see if we can find examples of both possibilities. Hm. If a function node is checked against an incompatible type, the body will definitely be checked against a failing node, and may synthesize a failing node. And if a node synthesizes a failing node, and is wrapped in an ascription, then it's the other way around. Well, that's inconclusive.

Anyway, there's some other things I wanted to check. Because of ascriptions, synthesis has to happen before reduction. This is only relevant for higher-kinded types. As it stands, this means that synthesizing the kind of an operator application may fail if it's partially reduced before synthesis, but, um, partial reduction indicates that typing failed, so the biggest concern here is whether the failure message ends up being confusing. In general, I believe we want to always check kinds before performing type reduction, so the question is whether I hold to that.

...

I do not. So, first major question: Should I be checking type kinds before performing reduction? If the answer is yes, it appears I only need to edit the base classes.

In any case, the other major question I have is whether I properly understand the operator types and kinds. Now, kinds can only subsume by equality, so that's nice and easy. What I need to figure out is, have I properly grasped the subsumption/"subtyping" behavior for the relevant types? In other words, is it possible that some kind of compound value type can result in subsumption checks between two non-proper types? Let's check the reference on bounded quantification.

...

Not sure. Let's check the code.

The type must be synthesized from a value, so the overall value type is proper. To contain further structure, it must be a thunk type. The only computation types that can be well-kinded despite dealing with non-proper types are universal and existential types. Maybe just existential types.

So, an existential type consists of a type parameter, a kind, and a type that ideally has the type parameter as a free variable. We need that this type is proper, if the type variable is bound to the specified kind.

So, it could be the case that the value type is of the form list[T], which might have a structure very much like an operator application around an ascription around an operator. (Operators would only be strictly necessary if it were doing something like passing in different container types for the variable, and applying them to a constant type.) The structure could be arbitrarily obfuscated. What I want is that if, for all T, the existential types under consideration evaluate the value types to compatible types, then the subsumption holds.

Now, once we get here, we're guaranteed that the types have been reduced before we do any checks for subsumption. (This sounds like it goes against what I was saying before, but that was about kinding; this is about typing.) It should be the case that the only subsumption checks we care about between operator types are for the operator type itself, though we can throw in operator application checking as a treat.

In fact, it may be necessary, because the only way I see to handle existential types, universal types, and operators, is to substitute the type parameter with a fresh type variable, and covariantly check for subsumption.

In the end, I've got a few things to look into:

Am I doing reduction and kind checking in the wrong order in the base classes?
Do I have the right approach for subsumption checks between types with type parameters?
Is there a good way to force subsumption to succeed if either type/kind is a failure node?
Failing that, is there a direction for checking subsumption that makes much more sense than the alternative?

Anyway, it's late and I need to get to bed half an hour ago.

Good night.

Weekly Roundup 2026-01-27

2026-01-27T05:00:00-05:00

Wednesday: I made some progress on Impliciula, and also on other things, but there were some issues I didn't see a way around.
Thursday: I described the state of another project, which I'll try to come back to at some point.
Friday: I made a little more progress on Impliciula.
Saturday: I fixed most of the problems I was running into with Impliciula, thereby allowing me to run into exciting new problems.
Sunday: I made slow and steady progress on Impliciula.
Monday: I ran into an ergonomics issue, and wasn't sure whether to try to fix it by implementing subsumption or overriding equality.

Next week, I'm going to try to implement subsumption. This is going to require some care, because it involves every type and kind, and potentially every type checking method.

Coding 2026-01-26

2026-01-26T05:00:00-05:00

Hm.

Hmmm.

I'm running into some issues here that I'm not totally sure what to do about. Basically, I came up with this whole fancy "pass a function to handle fresh variable bindings" thing for the helper functions for basic simply-typed lambda calculus. It would be really nice to do something similar with the various types introduced by quantification, but right now, that doesn't work, because the type variables being used are part of the type. Due to the way the universal and existential types are defined, right now the types have to match up completely. Where I currently do an equality check, it would be pretty much trivial to perform a substitution, though I'd also need to make sure to make sure that the type variable I'm substituting in isn't a free variable of the existential/universal type.

However, if I want to go this route, then I need to actually implement subsumption, because suddenly these types are "equivalent" without being literally the same.

Although...

Since what I'm going for currently is actually completely symmetric, I could also keep on calling it "equality", and simply redefine equality and hashes in some way... Not a big deal for equality, potentially somewhat cursed for hashing.

Also, I'm currently in the middle of rewriting the universal types tests; if I'm going to have any confidence that I'm updating it correctly, I need to do things in the order of:

Finish updating tests
Update equality, hashing, and type inference/checking
Once the tests are passing, it's time to update them again.

In any case, whether I do the later steps or not, I do have to finish the first one, so let's see about that. Later, since it's late now and I need to wrap up.

Good night.

Coding 2026-01-25

2026-01-25T05:00:00-05:00

All right. I was kind of taking things easy today, so I didn't do much. I just now updated a few more tests, and then cleared up some verbosity in the main code.

It's good that I'm chipping away at this stuff, though not very interesting, and right now, like right this moment, I really need to get ready for bed, because it's late.

Good night.

Coding 2026-01-24

2026-01-24T05:00:00-05:00

Okay. So.

I'm midway through making a change to the codebase, and I need to slow down a bit and see if I should make any course corrections. Basically, here's what happened:

I made a few base classes covariant, which allowed me to delete a bunch of code and addressed some problems I was anticipating with my efforts to rework the modules representing the different parts of the lambda cube.
That rework has several prongs. One is that I want to encode the node typing directionality into the type system, even though that can't be part of the syntax tree itself. Another is that I want to make it a little easier to work with variables. Finally, at least for now, I'm doing this in the form of static methods on helper classes, so I can reuse them via subclassing.

I've got the helper classes together, but I'm not totally happy with the way this is requiring the tests to be updated. I think some of the changes are improvements, but I'm not sure how to do some of the updates. The big user-facing change is that nodes with a variable binding are created by taking a function as an argument, that accepts a variable and returns the scope where the binding is live. This means that the user doesn't need to juggle the different data types to set things up for these nodes.

Anyway, I don't have anything else to consider or talk about right now, so I'm going to wind down.

Good night.

Coding 2026-01-23

2026-01-23T05:00:00-05:00

I'm poking gingerly at the idea of making helper classes for constructing nodes that I can use as mixins. There are two basic issues that I'm aware of right now:

How do I get the correct type onto base values and types (that is, the correct set of features)?
How best to make some of these helper functions usable as decorators?

I've got some stuff I'm trying for the latter, though I need to wrap up for now and get back to it later. Some ideas for the former as well, but they don't feel fully baked yet.

Anyway, really need to get to bed.

Good night.

Diary 2026-01-22

2026-01-22T05:00:00-05:00

I'm poking a little at Impliciula, but not sure whether I'm doing things right, so I'm going to just trundle along at it, not rushing.

So, the other thing I mentioned thinking about.

I've been watching some videos on tetrachromacy, and it inspired me to investigate the vision of natural tetrachromats, specifically birds (and most specifically some hummingbirds, I guess). Judging by the diagrams I'm looking at, which may be somewhat schematic, bird cone cells have more evenly spaced peaks of activation than human cone cells, and possibly narrower ranges of activation? Experimental evidence, (and some extremely simplified toy models I sketched up), supports the hypothesis that these hummingbirds, at least, perceive tetrachromatic colors.

Now, some of the videos I watched concerned efforts to devise color names for the various non-spectral colors that the research featured in those videos are supposed to make possible, but I wondered, if a bird were to devise color terms, and gradually, rather than all at once, what would it look like?

I've just scratched the surface in thinking about this so far, and I definitely need to come at it more systematically, but what I have so far is, I believe it's plausible that the first color term beyond black and white would correspond to red, ultraviolet, the colors in between (a sort of un-blue ultra-magenta), as well as some colors around red, and a broader halo of colors around ultraviolet. My basis for this is that both colors have very significant meanings for birds, and the non-spectral combination appears to be one possible coloration for flower petals.

Because the tetrachromatic hue space has a different structure than trichromatic, I'm not sure how best to divide it up further, or even how to make sure that I'm getting everything. I can consider, for example, the equivalent of the CMYK system, but I'm more interested in the more ad-hoc earlier systems, stuff that relates to the values of a hypothetical bird-people society, rather than "here are the five colors of ink that they need for their printers, when (if) they invent printers".

Anyway, that's all I've got for now. Going to sleep, and see if I come up with any more ideas or thoughts on... like, anything.

Good night.

Coding 2026-01-21

2026-01-21T05:00:00-05:00

I've been putting most of my mental energy into things other than Impliciula or conlanging, but I don't have the time or focus to write that up right now, so I'm going to jot some quick notes on what I've done and where I'm going with Impliciula.

I created a module containing a wrapper class with an additional parameter to track type inference direction. (To get this to work, I had to specify the variance manually, which now has me paranoid about where else the variance inference could be wrong.)
I was briefly imagining using this class to define helper functions in the same modules as the different node classes, but I've decided this would be frustrating, on account of how I'm no longer importing variable nodes, but I am importing variables.
Perhaps what would make sense would be to add constructor methods to the different node classes, that use the wrapper class, and then elsewhere have the helper methods that do stuff like synthesize variable nodes to pass into a function mapping variable nodes to computations.
I'm still not totally sure what is the best way to package up these helpers for reuse. I keep thinking that multiple inheritance in a mixin kind of mode should work, but then I get caught up trying to figure out the type variables, and I'm just not sure.

Oh well, I can at least do some of this, incrementally, and it will function as nice documentation, if nothing else.

For now, it is super late and I need to wind down.

Good night.

Weekly Roundup 2026-01-20

2026-01-20T05:00:00-05:00

Wednesday: I made no progress on the conlang code, and some progress on Impliciula.
Thursday: About the same.
Friday: I finished one bit of work on Impliciula, and prepared to pick up another.
Saturday: I got tangled up in a weird Mypy bug that I think is actually somehow several smaller bugs in a trenchcoat.
Sunday: I finished another bit of work on Impliciula, then decided to try and reorient a bit.
Monday: In an effort to get inspiration for Impliciula's code, I tried to work on its documentation. Things did not go as planned.

Next week, I may need to drastically change direction to get my brain feeling better. I am aware that this will further distance my Conlang Year stuff from anything resembling a "year", but I need to work with the brain I've got, not the brain I want.

Coding 2026-01-19

2026-01-19T05:00:00-05:00

Well, I didn't make any progress on Conlang Year. Instead, I ended up trying to document Impliciula('s IR) in an effort to get an idea of how the code should be laid out. The result of this so far is a really bad effort to cover a graduate course in computer science, really quickly.

I need to get some downtime somehow, both in the sense of getting to bed once this publishes, and in the sense of switching gears to something that doesn't stress me out so much.

Good night.

Coding 2026-01-18

2026-01-18T05:00:00-05:00

I've got a messed-up Mypy bug in, and people can have fun with that.

After I got some sleep, I realized that the way I'd written the code to hit the bug was a lot of extra effort for no gain; so I unwound all of the "clever" stuff, and the code got a lot shorter.

I spaced out a bit on writing this entry after that sentence, so let's try to get it together and write something up quick.

So, I have an intuition that it would be nice to define function nodes by passing a constructor a lambda that maps a fresh variable to a computation. (This is in some ways problematic, because, following f-ing modules, the variables don't freshen.)

The other side of things, which I hadn't thought about for a bit, is that it would be nice to represent typing directionality in a builder context (since reduction doesn't preserve typability), so there would need to be some kind of wrapper class to represent that. Given the experiences I literally just had with prematurely implementing things, I definitely at least need to sleep on this.

Since it's late anyway, I'm going to just post this, and for now just write up the directionality of all of the relevant attributes; this should illuminate something or other without committing to any specific code change just yet.

Better wrap up after that.

Good night.

Coding 2026-01-17

2026-01-17T05:00:00-05:00

I just switched on the usage of the Failure feature, and it was certainly... an experience. I ended up pulling out a lot of # type: ignores, some of them seemingly for a bug in Mypy, which I'm going to have to figure out how to minimize.

So:

Isolate Mypy bug.
Create DSL classes to hopefully make imports less obnoxious in tests.
Write more tests.

Probably in that order.

Anyway, bed.

Good night.

Coding 2026-01-16

2026-01-16T05:00:00-05:00

Okay. So. I just finished up with the big changes to the node definitions. Things are generally simpler, although there are a few unresolved questions kicking around. I believe my next step is to tag the failure nodes with the failure feature, which will cause a bunch of things to break, and when that breakage is resolved, I'll be ready to consider questions like "Okay, seriously, how should these modules be laid out on disk?"

But there's no way I'm considering any of this tonight, so let's wrap up.

Good night.

Coding 2026-01-15

2026-01-15T05:00:00-05:00

Okay, still pondering the right way to handle the conlang stuff; I know I shouldn't rush ahead, because I'm still refining stuff.

On the Impliciula side, I finished teasing apart some things that shouldn't have been together, and removed a bunch of weirdly-placed imports; getting rid of the remainder will take some very specific choices of layout.

I want to keep going on any of this, but it's late, so I'm going to wind down. Reluctantly.

Good night.

Coding 2026-01-14

2026-01-14T05:00:00-05:00

Okay, it's late, so, going through things quickly:

I haven't yet figured out the right way to encode paradigms to make agreement likely/feasible for fusions, so the conlang code is just kind of sitting there.
I shaved off a few lines from Impliciula by converting some values to computations.
I separated out variables and variable nodes, thereby unleashing absolute chaos. I think I've improved the code, but I can't really establish that for sure until I fix all the tests that I just broke.

That's not happening now, because I should have gone to bed fifteen minutes ago.

Good night.

Weekly Roundup 2026-01-13

2026-01-13T05:00:00-05:00

Wednesday: I let out an anguished scream of incoherent rage, and stopped screaming afterwards, because constantly screaming is bad for your throat.
Thursday: I tried to get back to my various current projects.
Friday: I looked at Impliciula with fresh eyes, and didn't like what I saw, but there's definitely potential.
Saturday: I did some more planning work on the projects.
Sunday: I made plans to improve the Impliciula code by making it less clever.
Monday: I figured out more work that I have to do on the conlang code, and I think after I published, I made some breakthroughs on Impliciula.

Next week, I'm going to put those breakthroughs into practice, and it's going to have so many weird cascading changes.

Coding 2026-01-12

2026-01-12T05:00:00-05:00

I made some localized improvements to the Impliciula code, and fixed saving in my editor by, um, upgrading my LSP? Modern development is weird. Anyway, I'm having trouble figuring out what to work on next, because the code right now is a confusing mass of circular imports, and I don't see any way that my planned rewrites will improve the situation, so that's messing with my motivation a little.

Given that, let's make some improvements to the conlang code.

...

Okay, I've added a new field type possibility, written a deserializer for it, updated the existing deserializer, and now I need to work out how to process this new data in the fusion code. The remaining work appears to all be in the innermost loop, for certain definitions of "loop" and "innermost".

However, to get this done well, I'd like to have some helper functions. But it's a little hard to say exactly what these helper functions should look like.

The highest-level function I need would take a mapping of lexicon entries, a part of speech, and either a root or a specific inflection. If it's not a specific inflection, and the part of speech matches, then it needs to iterate over the inflections present for that part of speech; otherwise it needs to return X copies of the given inflection, if any, or the bare root, where X is the number of inflections associated with the part of speech.

Writing that out suggests that I should track inflections separately from any given entry. That's another change to deserialization and serialization, and a slightly fiddlier one, so I'm going to make sure my current changes haven't broken anything, and come back to this later, since it's getting late.

Good night.

Coding 2026-01-11

2026-01-11T05:00:00-05:00

I mostly thought about stuff today. Here's my current stumbling block:

I want to statically prevent failure nodes from occurring in an AST. (This may be infeasible.) Currently, my solution to this is "Don't do that :)" and I'm having trouble improving on that.

Basically, something in the current and planned implementations isn't quite right, but I'm not yet sure what.

The general idea is that failure nodes are an expected consequence of the coercion mechanism, which means that they can arise in type checking, but I'd like to guarantee that they aren't in the syntax tree or inferred type, should type checking succeed. The barrier I have to this, is how to properly represent failure nodes. If I represent them as a "feature" (which is a streamlining of the current system, which works, but has amazingly bad ergonomics), then I need some higher-kinded-types magic, or possible generic associated types magic, in order to get the type I actually want.

Before I distract myself thinking about what type magic I do have access to, let's consider how to simplify things.

The most straightforward simplification would be to simply provide an argument to coerce to tell it what to pass back. This wouldn't be too big of an ergonomics hit, and it might allow for some improvements around record types. So, let's say this is definitely on the table.

The next issue is that there's a soundness hole in the current implementation, which allows for some bad casts. Practically speaking, it would be very weird for this to actually be relevant, but it bothers me. All the same, it seems likely that the correct solution is basically to slap a note on the function saying "if you call this function outside of some very specific contexts, that's on you".

That's definitely enough thinking for now, better actually wind down.

Good night.

Coding 2026-01-10

2026-01-10T05:00:00-05:00

I think I've fixed the issues in my plan for fixing the conlang code, so I'm going to sleep, see if any more objections occur to me, then start updating the code.

For now, I'm trying to give the same sort of treatment to my planned rewrite of Impliciula. Here's what I'm thinking. Sub-packages for STLC (always available), failure, universal, existential, operator, and record. Each sub-package defines a marker class in a module with no dependencies, puts its nodes in as many modules as feels right, and provides a mixin class for constructing nodes without having to bring in the node classes. Then, the different sets of allowable nodes can be specified just by combining the marker classes and mixins.

The other side of this is, I feel like there has to be some way to parameterize the allowable scalar node types, but I could just be wrong about that. I'm not currently doing it, and not trying would reduce the implementation effort, so...

Anyway, I just noticed that one wrinkle is the typing around coercion. I want to make failure types be distinct, which means that the Failing protocol needs to somehow account for that, and how it can do that is not obvious to me at this time.

By the way, "this time" is pretty late, and I should get to bed.

(I think the answer for now is to just do a bunch of overloads or something like that.)

Good night.

Coding 2026-01-09

2026-01-09T05:00:00-05:00

Okay, I'm letting stuff for the conlanging code kind of just percolate.

Let's take stock of Impliciula. There's some functionality in here that got kind of messy, and it's clear that I didn't think through how it should be used.

The basic idea is, there are different extensions to the simply-typed lambda calculus, and I managed to write code that can represent different extensions in a typed fashion. So, like, it's possible to have basic simply-typed lambda calculus, system F, system F with existential types, etc. Each node is only implemented once, and the specific combination of allowable nodes is handled in a module for the specific extension of the lambda calculus. The problem I have right now is that I don't have a clear picture of what should be allowed by default. Because the current state is so disordered, I'm going to try to lay out how things should work.

The base STLC is CBPV with bidirectional typing. This means that it needs variables, thunks, forces, returns, sequencing, abstractions, applications, and ascriptions. Furthermore, higher-level features require type variables and the kind of proper types, as well, and typing requires several "failed" nodes, which are in kind of a confusing place.
In order to actually compute an externally useful value, basic scalar types are needed, at minimum.
System F extends with type abstractions and type applications.
Additionally, packing and unpacking.
In order make use of packed values, record types and projections are needed.
Lastly, type operators and operator applications are needed for System Fω.

The idea of breaking up the nodes into distinct features, suggests a means of streamlining the current implementation: instead of the current approach of enumerating all (most) nodes of interest, have a single type parameter that can be unioned with a marker type representing a constellation of related features. This should simplify things; I'll need to make sure it doesn't somehow simplify "too much".

With this level of simplification, it could make sense to introduce more nuance into the base types. Make a given syntax tree module specify a combination of features and base types. Then the default can be CBPV and bidirectional typing, plus support for higher-level features, and the extensions can be base types, universal types, existential types, type operators, and record types. The resulting syntax tree types should generally be parametric over base types, since they don't do anything "interesting" to tree structure. Therefore, it could be clean to make this all a two-variable system, with one variable for node types, and one variable for base types.

As I think about this, it also occurs to me that so many imports would be simplified if I were to put together some kind of subclassable builder class, like I've considered before.

I think this is a promising direction to take the code, if a little obnoxious to do all of the updates. I just need to give things a little more thought to make sure they all fit together. (The current source of annoyance for me is, if typing fails, there could very well be a failed node somewhere in the result, but if it succeeds, then there shouldn't be a failed node. Do I need to thread a result type through all of type inference to get the results I want? I want the answer to be "no"... All of this said, nothing in the current types stops inference from returning garbage, so on the one hand, it could be defensible to do the updates I know how to do now, and the rest later, but on the other hand, I want to make sure I don't accidentally impede those later updates. I've got to think...)

Not going to get any further on this tonight, need to wind down.

Good night.

Diary 2026-01-08

2026-01-08T05:00:00-05:00

Okay, I'm... stabilizing. Let's try to get back to other topics.

Yesterday, I did do the Conlang Year prompt. Sort of. And I've already done everything in today's prompt. More or less.

So, let's see what's next.

...

Well. I'd already done pronouns. However, I wouldn't be opposed to adding a formality distinction, now that I'm reading over this.

Okay, like, even knowing that it's extremely likely that this prompt is just for thinking about what pronouns to have, I at least want to figure out a derivation for the formal version. And that will perhaps take some thinking, because my first thought was to copy Spanish, which has its formal second-person forms derived from possessive phrases, and the possessives in this language are, special. If I go this route, everything in this language that has declensions, will have regular declensions, except for this one pronoun, which will instead have a mess.

For a bit of variety, let's make these changes in the Jackal lexicon rather than the Eagle lexicon. This is early enough that the possessive preposition doesn't have its exact modern pronunciation, so there's a chance that sound evolution will do something weird. At the same time, it's not at the same time (as every other fusion I've done so far.)

One thing to note, which shouldn't be a problem in this instance, but may in the future, is that I need to figure out how to select a particular inflection when fusing. Actually, one thing that will be a problem is that I'm not propagating fusion changes to the children. I'll have to fix that.

...

That's not simple. I think I'll write all of this up in my notes, and ponder this. The big issue is really just getting the representation correct. After that, I just need to create some helper functions and restructure the fusion logic a little.

Anyway, I have to put this down for now, and get ready for bed.

Good night.

Diary 2026-01-07

2026-01-07T05:00:00-05:00

I started writing this shortly after the last entry because I couldn't get to sleep. The Trump administration is doing things so nakedly imperialist that it feels absurd not to say anything, but I'm once again wary of speaking outside of my areas of something vaguely resembling expertise. Counterpoint: I clearly have more grounded opinions than Chuck Schumer, so go off I guess.

So much of existing in America is being confronted with countless wars that are forced to be nameless, or to bear euphemistic names. Wars against people who were just minding their business, wars against people who are nominally American citizens, at least for now. Wars, wars, wars, constantly and everywhere, but we just hear that everything is normal and functional. Every predictable outcome of these wars is an unfortunate coincidence, or a clear consequence of poor moral character—and the victims must have poor moral character, for if such things were to happen to someone with good moral character, that would be monstrous, and we cannot be living in a world so unjust. (Never mind that this is monstrous regardless of the moral character of the victims.) Wars, wars, wars, attempting to wring out valor and glory, as though we could make something good or beautiful out of valor and glory, whatever they may be, however they are to be found in these daily atrocities, as if such absurd vanity, such poisonous fear of irrelevance, could bear wholesome fruit.

Every year of my life, the gilt flaked off of the previous year's golden idealism, but another shining vision of tomorrow arose to replace it. I have seen so many shining visions, I fear I'm going blind. Perhaps that is why I cannot see the next.

Not a reaction to a single event, but the inevitable consequence of so many such events...

I don't want glory. I have too clear an image of the pathology of fame, one of many poisons that has been sold as a tonic. I merely want the chance to create something that I can understand as good. But how can I trust it to stay that way? If I create art, will people I've never met nor heard of twist it to support new atrocities? If I create tools, will they be turned against the biosphere in the name of profit?

I know I can't give up, but right now, I can't see a clearer vision of moving forward than to simply continue, one step at a time, trusting my instincts to guide my path. It doesn't feel good, but it will have to be enough.

By the way, you should definitely read what actually informed people have to say about our entire multi-century situation. I'm writing totally off-the-cuff here, and I don't think any part of interacting with this blog post is praxis. (But I could be wrong. Again, seek out relevant expertise.)

Good night.

Weekly Roundup 2026-01-06

2026-01-06T05:00:00-05:00

Wednesday: I prepared to jump ahead on Conlang Year.
Thursday: I started thinking about Impliciula again, and working on it some.
Friday: I polished up the work I did on Impliciula.
Saturday: Tired, but still got stuff done for Conlang Year, sort of.
Sunday: A little more done on Conlang Year.
Monday: Ugh. Come on. Just gotta get through it.

Next week, I'm going to try to handle not being on vacation any more.

Diary 2026-01-05

2026-01-05T05:00:00-05:00

Okay, so, it's late and I don't have much time, which makes it convenient that I accidentally sort of already did the Conlang Year prompt I was planning for today. I'll try to keep on with the subsequent days, because perhaps I want to change how I did things, but for now, I'm just going to write up the current state of things. This is the stuff for demonstratives, by the way.

Anyway, I'm going to wind down now.

Good night.

Diary 2026-01-04

2026-01-04T05:00:00-05:00

All right, I have completed a day of Conlang Year, by thinking about things a little, and then writing stuff down. Good enough for me, and it would be wildly irresponsible to do anything more tonight, so I should stop doing things now.

Good night.

Diary 2026-01-03

2026-01-03T05:00:00-05:00

We traveled a bunch today, and are wiped out. I didn't work on anything, though I did check over exactly how much of Conlang Year I plan to skip. I identified some stuff that I'd tried to avoid, but actually there wasn't really a point. I could plan to do that tomorrow, or...

...

Yeah, I can basically just do it.

Although, I'm really tempted to mess around with the sound changes until a lot of what I did is just invalidated.

Well, I'll try to hold off until tomorrow. Right now, it's late.

Good night.

Coding 2026-01-02

2026-01-02T05:00:00-05:00

Okay. I kind of slacked on Conlang Year today because I already did the prompt. Probably to get caught up on work put in, I should come up with a bunch of example sentences so I can lexicalize stuff.

But for now, I mostly focused on Impliciula. I documented the classes I added yesterday, so that's all set, and now I just need to dig myself out of the hole I just finished digging, coverage-wise. I'm planning ahead some for testing existential types. This will need yet another new set of nodes that I didn't realize I was going to need so soon, this time based around record types.

Record types have been a bit of a bother to me, because I didn't quite understand why they have to be computations in CBPV, but simultaneously my implementation of CBPV plus bidirectional typing is pushing them to be computations for what can't be the same reasons. My main concern with this, beyond, "I don't get it" is that I'm not sure what the representations of these things will look like, but potentially this is all just a matter of giving thunks a variety of representations, depending on what type they contain.

Anyway, especially after last night, I really need to get some sleep.

Good night.

Coding 2026-01-01

2026-01-01T05:00:00-05:00

I mostly just thought about stuff for Conlang Year, in an effort not to rush things. So, what I was doing just now was fixing up the names in the syntax nodes for Impliciula, then I deliberately tanked my coverage numbers (at least to some degree) by implementing a draft of type operator support, even though I don't feel like writing the tests yet. I do not feel 100% confident that I got everything right, but it must be pretty close.

I need to get to bed soon, but I want to dash out some quick thoughts first. I was mostly working with existential types today, and in order to test them, I'm going to need record types. I'm going to need to check over some of the articles I'm basing this system off of, to see if there are any subtleties I should be aware of. I need to check my sources to see if I have the string representations right. Order of testing is, shore up any coverage not related to existentials or operators, implement record types and anything else I decide I need, write tests for existentials, write tests (and documentation) for operators, get coverage up, then do whatever tidying-up occurs to me, and work on the next layer up, which I have written in a notebook somewhere.

Anyway, time is passing at a terrifying rate as I write this, so I'm going to stop now.

Good night.

Diary 2025-12-31

2025-12-31T05:00:00-05:00

I did Conlang Year stuff, and then took a break by looking at stuff for Impliciula, without actually touching the code. Looking at the entry previews for the next week, um, I might have accidentally done a bunch of this already? So, I'll try to get something for Conlang Year done tomorrow, but I might end up skipping ahead. Also, fixing attribute names in Impliciula. Anyway, it's late again, so let's wrap things up.

Good night.

Weekly Roundup 2025-12-30

2025-12-30T05:00:00-05:00

Wednesday: I got basically back in the saddle.
Thursday: I made unimpressive but definite progress.
Friday: I did a few days at once because I got tired of mixing up noun class and noun case.
Saturday: I got going later than I'd like, so things were a bit rushed.
Sunday: I fell agould of the fact that this project is kind of an enge-lang awkwardly disguised as an art-lang, so many things that you'd think would be prepositions are actually verbs, or at least, they will be.
Monday: I got basically caught up.

Next week, I'm going to try to stay on pace and avoid further overcomplicating things. Maybe toss in some other projects.

Diary 2025-12-29

2025-12-29T05:00:00-05:00

Okay. Okay.

I've put together a list of adposition and adposition-adjacent words to form, as well as planned roots for most of them, aside from all of the verbs. There's more work to be done, but there's also sleep to be slept, and so the balance tips towards "wrap it up ASAP and get to bed".

Good night.

Diary 2025-12-28

2025-12-28T05:00:00-05:00

All right, I'm trying to figure out what prepositions I want. A complication here is that a lot of obvious choices, I actually want to express with relative clauses or something like that. The eng-lang side of this conlang is showing a bit. While there are a few prepositions I've admitted I need, my notes now have a bunch of places where I decide that some concept "goes in the locative hole".

...

All right, I've decided that there are a few more candidates for prepositions, but I'm not done considering things, so I'm going to have to try to finish this up tomorrow, then do the next prompt. It's later than I realized, better get things together.

Good night.

Diary 2025-12-27

2025-12-27T05:00:00-05:00

I cleaned up the dictionary mostly, and what's left needs me to add some capabilities to the code. To keep the momentum going, I'm going to try to cover adpositions tomorrow.

I did think about it a bit more, and now I'm unsure whether one of the adpositions I have in mind is actually a preposition, or actually an article, or perhaps one of the articles is actually a preposition. I think I can defend the distinctions I'm currently drawing, but the sentences I'm coming up with for that defense are... odd.

Anyway, I don't want to stay up super late, again, so I'm going to wrap this up now, even though I didn't really post anything.

Good night.

Diary 2025-12-26

2025-12-26T05:00:00-05:00

I realized today that I was doing multi-part prompts out of order again, so I did a bunch of prompts to my satisfaction just so I wouldn't be bothered by that for too long. What I need to do next is to review all of my lexicon entries and ponder how I want to present them. I've kind of cheated my way around a bunch of dictionary-related questions by avoiding thinking about presentation for now. I'm not planning to lock down presentation or actually implement it, but I've got enough data in here that it would behoove me to have some idea of how I want to portray it.

Anyway, let's see what I should consider, providing I turn on my laptop early enough tomorrow.

Fix short glosses: replace spaces with dashes, and generally move nuance into the long gloss
Furthermore, some short glosses are in need of correction. The demonstratives aren't marked as demonstratives, the locative copula isn't defined as the locative copula, and I haven't figured out what to call one of the pronouns, so the current gloss is just pretty wrong...
I want to mark many nouns with a preferred measure word.

That last one, it probably makes sense to mark them after the measure words exist, which means I need to figure out how to do more sophisticated semantic drift operations.

I really don't want to be awake too much longer, so I'm going to wrap this up.

Good night.

Diary 2025-12-25

2025-12-25T05:00:00-05:00

I took things a little easy and somewhat skimped on the prompt for today, but I did do it.

I tried reading my example sentences out loud, which...

I'm still the best person at speaking this language. That's how this works. But hoo boy.

Anyway, I want to wrap things up, because I think that will be better for me than obsessing over getting things just right.

Good night.

Coding 2025-12-24

2025-12-24T05:00:00-05:00

When I was trying to squeeze in work on the conlang workflow around writing the roundup (it end up going the other way around), I knew I was working with untested draft code, so I was ready for anything. Except for nothing. There were an astonishing number of ways for my code to simply fail to have an effect, from applying the wrong drift lexicon, to using the wrong metadata, to failing to update metadata.

Now that I've gotten through many issues (and I'm skeptical that I got all of them), the main thing I'm aware of is that the current design is super slow. Basically, right now it fires up a separate sound change process for each noun, which means that there's a slowdown of at least an order of magnitude. I can claw this back to almost the current speed using some kind of pooling mechanism, but this seems like scaling will overwhelm it. I think the "right" solution is to figure out a reversible way to intersperse the morphologically derived sounds in with the base sounds, in such a way that the structure can be recovered following the sound change. This could be done by putting the root pronunciations first, then traversing the children, and then using a custom iterator instead of zip. Let's see if I can throw this together quickly.

...

Okay, I didn't time it, but I remember it being much worse last night, and also the code is mostly clearer now, so let's move on.

...

Okay, after a bunch of back-and-forth with Brassica, I've gotten sound changes that do something. Not quite enough for my taste, but I need to take a break from it for a bit. So, let's try to remember where I was with the prompts.

Ah, I was on day 52, and I was kind of laser-focused on the case stuff, which means that there's other stuff that I'm sort of behind on. I'll have something workable to start with if I can fill out all of the roots I still need.

...

Okay, the roots are filled out, but I forget what to do for the derivations. I'm going to let this sit for a while tonight.

...

All right, let's take a look at the current state of things. I know that I'm deriving these different words from nouns, and I want to perform the derivations before the declensions, but after fusion. With the current (dire) state of things, that means that I want to run a "particle filter" drift lexicon on eagle-lexicon-combined.toml, along with some form of erosion, creating eagle-lexicon-particles.toml, then include that into eagle-lexicon-combined.toml to create eagle-lexicon-with-particles.toml, which should be used as an argument where eagle-lexicon-combined.toml currently is.

...

All right, I slapped on a low-effort word change, and it mostly worked, but I uncovered some more breakage in the existing sound changes, so now I have to fix that.

...

All right, that's enough "all right". I patched up the sound change stuff, and there's more work to be done, but I'm well enough through day 52. I peeked ahead to day 53, and I can do that tomorrow, except that the noun inflections are a little unpredictable, which was kind of the point, but it makes writing up the specific declensions hard.

Anyway, I want to wrap this up.

Good night.

Weekly Roundup 2025-12-23

2025-12-23T05:00:00-05:00

Wednesday: I had a mix of solid and weird ideas.
Thursday: I noticed that the stuff I'm trying to do requires even more new code to be written.
Friday: I updated the data format to accomodate my ideas. Technically nothing should have changed, but it does serialize differently.
Saturday: I didn't feel like touching code, but I was okay doing a little planning.
Sunday: I started putting one of those ideas into effect, that of dividing the sound changes into named eras.
Monday: I poked at the era stuff a bit more, and planned out the files I need to handle case endings.

Next week, after spending an embarrassing amount of time getting the tech side of this working, I'm almost ready to get back to the prompts. I just need to come up with some sound changes to cut down on how unwieldy the case endings currently are.

Coding 2025-12-22

2025-12-22T05:00:00-05:00

Once again, I mostly took things easy today, trying out a bunch of games from my backlog, but I did get the "era" divisions a little more done. Just a little. I don't feel like doing the rest of the changes right now, so I'm just going to reflect on what I know remains, and see whether I think it'd be enough.

Hm. Fundamentally, what's missing, and what I'm hoping the eras provide a baseline for, is a regular system for naming the different lexicons, rules, and input data files. "Just make everything into a monolithic script" sidesteps this by having one unique file per era.

But nothing is striking me as a satisfying solution to this yet, so let's just plan out the next steps in actually getting back to the prompts.

I need additional lexicons between eagle-lexicon-combined.toml and jackal-lexicon.toml. This is done by evolving eagle-lexicon-combined.toml to eagle-lexicon-cases.toml, via a set of sound changes that either shorten the roots, or do nothing at all, but filtering with eagle-case-filter.toml, which remaps the source roots to case markings, and I just realized that the paradigm implementation isn't fully ready yet, insofar as "how do you take stuff that already has a paradigm as input?" Possibly the answer involves making drift stronger so it can "pick out" specific affixed forms. Anyway, then eagle-lexicon-combined.toml has to be updated to eagle-lexicon-declension-initial.toml using eagle-lexicon-cases.toml, then updated to eagle-lexicon-grammaticalized.toml via eagle-grammaticalize.bsc. This is the new input to forming the beginning of the jackal era.

Since I don't even feel the urge to finish putting the rest of the stuff that I have into the era folders, clearly I need to put in more until it gets obnoxious enough for me to decide on a plan of action.

Not happening tonight, though, and tomorrow is a bit busy. I should wrap up quickly, for now.

Good night.

Coding 2025-12-21

2025-12-21T05:00:00-05:00

I didn't have much time to work on stuff today, but I was able to start doing the era naming, and splitting up sound changes. Splitting up the sound changes was nice, because it meant I got to delete a bunch of categories (... after duplicating everything). I just created the era folders; once I have the sound changes split up and the era-specific data all moved to its own folders, I can try to organize things better, and see if I have any ideas for improving the build pipeline. One thing I'm wondering about is getting more structure under the build directory; I might need to improve the command-line argument handling to make that feasible.

However, I should be focused on getting back to the prompts. What's in the way of that? Well, I feel bad about adding files to an undifferentiated mass, like a big junk drawer. So, I should re-evaluate my feelings on this, once the eras have everything divided into several little junk drawers.

Anyway, it's late and I'm tired.

Good night.

Diary 2025-12-20

2025-12-20T05:00:00-05:00

This week was rough, so I'm just going to take things easy.

As far as getting back to the conlang prompts goes, I don't have the repair strategies for forming the case endings ready, and before thinking about it, I want to figure out what to do to clean up the build process.

The two options I have in mind currently are to do the "eras" split like I've been planning, and to then consolidate all of the behavior and data flow into a single "era" script per era, which only works with lexicons. The other option is to try to organize the source files and the intermediate files so that the relationships between them are clearer.

The unfortunate conclusion is that, no matter what I do, the first step is the same, so there's no reason not to do it except that I don't feel like putting in the work.

I'd rather take things easy right now and try to get things together tomorrow.

Good night.

Coding 2025-12-19

2025-12-19T05:00:00-05:00

Okay, I've got a bit of time tonight, let's see what I can make of it. The big issue right now is that I have a lot of code that's making assumptions about the in-memory representation of the lexicon, and that's going to need to change. Currently, the values in the mapping are tuples of frozensets of pronuciations, and "metadata" objects. Now, thinking about this, I may actually be able to avoid thinking about this, because the children certainly seem to be part of the metadata. If there's metadata, there can be children, and covice coversa.

Currently, the children mapping goes from gloss extensions to pronunciation sets, but just to make sure I've future-proofed this, let's look ahead. In the event that a derivation creates "sort of a new word", then the choice of type consistent with current usage would be... a tuple of pronunciation sets and metadata.

...

Okay, I've updated metadata serialization and deserialization, and now that part of it should just work. Now I need to get the paradigm code working, because I've just thrown a big ol' wrench in all of that. ... To be fair, I just realized this code was sort of buggy anyway. Really buggy, actually.

...

All right, I've updated the code. I believe it works now, but I believed that before, and I was clearly wrong, so I'm going to wrap things up for tonight, and make sure I'm well-rested when I have to debug this nonsense.

Good night.

Coding 2025-12-18

2025-12-18T05:00:00-05:00

Oh boy. After I published the last entry, I realized there's way more that I'm doing wrong. In particular, currently I have a separate lexicon entry for each specific gloss. And I'm applying semantic drift, which changes the glosses. Which is going to get extremely messy once I start generating declensions. I'm not going to make four or five versions of every word in the drift lexicon.

The way that I think makes sense to deal with this is to create a trie-like structure representing derivational morphology.

Given an uninflected gloss, you have the pronunciations, the long gloss, the part of speech, and the tags.
Additionally, you have a map of gloss inflections; these can minimally just contain pronunciations.
However, if we have a derivation that makes it "sort of a new word", then under that inflection, we have the pronunciations, the long gloss (important for clearing up subtleties in the derivation), the part of speech, and the tags, as well as the appropriate derivations for that part of speech.

One obvious question is where to stop, and I think that has to be decided on a case-by-case basis. If a particular derivation is not used much, then it can be omitted from the lexicon proper, and derived via regular processes when needed. One of the processes needed here is to "re-parent" a sub-trie to the top level, if the derived form takes on an independent identity. Thinking about the right way to use these capabilities, my sense is that even one extra level of nesting should be used sparingly, but I'll have an idea of when it's needed. At the same time, the prompts aren't covering these kinds of derivations yet, so it may be a sensible use of my time to only implement enough to handle declension and conjugation. Although, I would like to make sure that I don't somehow make it hard to implement.

Fundamentally what we have here is a structure of nodes and leaves, where nodes and leaves contain data, but different data types. For serializing the data, we can have a field specifically for containing the children, which maps representations of inflections to the required data. I can implement a simple object layout that serializes identically to the more advanced possibilities. So, I've got a plan. What I don't have is any more time tonight. I'm going to wind down now.

Good night.

Coding 2025-12-17

2025-12-17T05:00:00-05:00

Okay, so. I was playing user-created level packs for Gentoo Rescue. So I didn't actually get anything done here. Let's quickly catch up on the state of things.

I've come up with the idea of dividing the linguistic evolution into "eras", and my sense is that it would make it easier on me if there were some kind of shorthand way to represent them, and my idea there is inspired by reading about BASIC as a child, but using words instead of numbers. So I could have eras like "eagle, jackal, orangutan, tapir", and then I can slot in more between them in alphabetical order if I need to. I don't know why I made them animal-themed. Anyway, each era would contain various folders, and I think if I'm going to have a good time here, I need to figure out how to make the file structure totally regular and consistent, because my gut says I'll have a terrible time if I try to generate my build file based on the shape of the directory tree.

Perhaps the dynamic structure needs to live in the data files themselves? This would represent two things: yet another rearchitecture, and specifically storing the sound changes in TOML, which is somehow less cursed than it sounds, but only a little.

Right now, I'm pondering the idea of writing a stack-based language that reads commands in the form of a TOML array, and that sounds cursed enough that I definitely need to take some time to think about this, rather than rush to prototype. I guess one question is whether I can just take my existing Python code and call it from scripts as a module. That shouldn't be too much worse to use, and it's less bad to implement, I think.

Definitely don't rush to implement, but do rush to bed.

Good night.

Weekly Roundup 2025-12-16

2025-12-16T05:00:00-05:00

Wednesday: I made some progress on conlang tooling.
Thursday: I tried to make some progress, but eh.
Friday: I worked out more of the code, and presumably wrote something, based on the category I put it in.
Saturday: I worked out the broad strokes of how I plan to do sound changes with affixes, and therefore got some idea of how to implement the relevant commands.
Sunday: I figured out exactly the command implementation I want, so, like, why not write the code?
Monday: My build system code is bad, so I really want to fix it.

Next week, we'll see what I manage. I'm so tired right now, and it's not even dinnertime yet, so this week is already off to quite a start.

Coding 2025-12-15

2025-12-15T05:00:00-05:00

Okay, I just quickly typed out build rule generators for paradigm calls, and now I'm staring at the existing rules like, okay, I have to come up with more structure for this, my filenames are just nonsense.

If I were to push ahead with this, I would need to filter and rename from build/proto lexicon fused final.toml, and then use the resulting affix lexicon to create an "initial" lexicon file, then apply a new set of sound changes to create the "final" lexicon file. When I work on aspect markers, that's just another paradigm step; it can just go before or after setting up the case markers, and then everything can get sound changed.

Now, while I'm planning ahead, one of the other things I want to support is converting nouns to prepositions. This is done via change + drift + filter, then loan back in.

It's late, so I don't want to touch the code any more tonight, and I need to sketch out some potential directory structures, because I'm having trouble developing the ideas entirely in my head.

For now, got to get ready for bed.

Good night.

Coding 2025-12-14

2025-12-14T05:00:00-05:00

Okay, I'm writing late again, so let's see if I can bang something out quickly.

My hope is to get something for this paradigm stuff, and I want to avoid thinking too hard about what features I could possibly have for it, so let's look at the immediate stuff. I think it's reasonable to call case and aspect markers a part of speech, so the combination here boils down to selecting two parts of speech. Like, a noun and a case marking. I'd been thinking of doing this with multiple lexicons, but now that I'm thinking about it, I'm a little unsure how this should look. There's a main lexicon that gets updated, and any number of auxiliary lexicons to add more stuff, and selectors, which right now are any part of speech. If I do add them right back into the output lexicon along with everything else, then I just need to make sure that any post-processing only operates on the word boundaries. Adding them back does make sense, because in some cases, the original root will be removed.

Okay, in talking this out, and thinking some about other things, I've put together a formidable set of command-line options. Let's see how to use them together.

The input and auxiliary lexicons are scanned for instances of the affix part of speech, resulting in a lexicon containing a single part of speech.
Each gloss is validated to either start or end with a hyphen.
A method is called on the input lexicon, with arguments of the affix lexicon, the root part of speech, and whether to remove the matching roots.
The resulting lexicon is written to file.

I think I see how to wrangle this, the question is, do I feel like doing that right now? Not... really. Well, I'll see about filtering and validating.

...

Okay, the validating is kind of overly clever, but I can just print the entire lexicon if something goes wrong, so I'm safe to just do the stuff I'm doing.

...

All right, I got carried away and coded the whole thing, really jankily. It's probably embarrassingly broken, but instead of thinking about that or trying it out, I'm going to get ready for bed.

Good night.

Diary 2025-12-13

2025-12-13T05:00:00-05:00

Hoo boy.

Rough end to a rough week. I didn't touch any code or other text, so I'm just going to jot down some thoughts on what I want to do next.

I'm trying to figure out the right way for me to handle affixes. In writing erosion rules for compound words, I tried to spread the deletions out evenly, but my impression for affixes is that I want the root to not be too changed. So, like, assimilation rather than deletion, but the affix is fine to reduce, as long as different affixes still contrast. Now, one of the wrinkles here is that I want some of the aspect markers to be prefixes, and some of them to be suffixes. I think it seems brittle and unpleasant to construct and simplify two related sets of affixes in parallel like this, so I think what I want to do when generating affixed forms is to, at the word boundaries, mark the affix vs the root. There's a section in the Brassica documentation on how to do stuff like this; it seems pretty simple. With that marking in place, I can restrict changes to apply to only the end of a prefix or the beginning of a suffix, or whatever.

I'll see if I still believe this in the morning. For now, I'd better wrap up.

Good night.

Coding 2025-12-12

2025-12-12T05:00:00-05:00

Okay, I've got a weird and tangled mess of files that produce adequate pronouns. And maybe I'm going to put an animacy distinction in the pronouns? I kind of don't want to...

Anyway, that produces most of the data that I need to create case endings, but I don't have any of the code. I kind of want to force all of this into being, both so I can see the case endings in practice, and also so I have an idea of the full scope of Things That Will Be Going On In Code, because wow it just keeps getting messier. And if I have the machinery around for creating case markings, that should give me enough information about how it needs to be possible to lay stuff out.

For now, though, I'm going to accumulate tech debt, and then try to balance dealing with it, with Actually Doing The Prompts Again.

For now, I need to get ready for bed, and get one day closer to dealing with this awful week.

Good night.

Diary 2025-12-11

2025-12-11T05:00:00-05:00

Ugh.

Not a productive day today. I can see exactly the steps I need to polish the pronouns in the proto-language, and I'm currently running on zero inspiration for the sound changes. Like, all I've got is "better delete stuff". I think I'm going to need to take some time (which I currently don't have) to just say these words over and over and figure out what kinds of changes I want.

(I mean, with just three words, I could just specify new pronunciations by fiat, but I'm going to want something very like these changes to form the case markers, so it's vaguely justifiable to try to get some general principles.)

Anyway, spaced out too much, sleep now.

Good night.

Coding 2025-12-10

2025-12-10T05:00:00-05:00

All right, let's see where I am. I've got a bunch of compound roots that I need to fill in, grammaticalization to do, and some disappointing stuff so far. Let's try tweaking a root just to make stuff happen...

Okay, I got a little irregularity from a tiny tweak. Let's avoid iterating on that for now, and focus on the fundamentals. I need to form some of these roots, even if my plan is to immediately subject them to grammaticalization. All right, here's the sequence:

Form personal pronouns, and shorten them
Now, select out the roots for affixes, and create new glosses with shortened versions of them.
Now, implement procedural combination
That produced two-word entries, fuse them

There are some problems here that need to be addressed, and I haven't decided what's the best way to handle them. Basically, right now, compounding roots (which is how the pronouns are formed) "have to" go back in the full lexicon, so to apply changes exclusively to them, I need to either remove them from the lexicon, or allow overwriting... I think I prefer overwriting.

All right, I've got a plan, and I've got code. I don't have the sound changes. I'm going to try to take things easy, and wrap up for today.

Good night.

Weekly Roundup 2025-12-09

2025-12-09T05:00:00-05:00

Wednesday: I collated some information for conlanging.
Thursday: I planned the code side of things for what the prompts are asking for.
Friday: I fixed the data format that I'm storing the lexicons in. Again.
Saturday: Didn't get much done.
Sunday: I realized that I'd been doing prompts out of order, but, like, it's fine probably.
Monday: I stopped touching the prompts in favor of getting my code working.

Next week, I'm going to try to finish writing the code that I need right now, then iterate on the lexicon and sound changes a bit. The results from the code I've got written right now are pretty lackluster, which isn't what I want from the sources for case endings, I don't think.

Coding 2025-12-08

2025-12-08T05:00:00-05:00

The prompts are paused while I try and shore up my root generation and sound changes. I've got an initial shot at overhauling the generation, and the sound changes are a little frightening to me. I think I'll post now, and try to wrangle with it later.

Good night.

Diary 2025-12-07

2025-12-07T05:00:00-05:00

Okay, so. I realized today that I got a little confused with some of the prompts, and did stuff out of order. Since I was so tired today, I'm going to not stress too much, and try to fill out a few words that I haven't settled on yet. Afterwards, I'll get more done, but I just want to get this entry out of the way so I'm not worried about it.

The sooner I can wrap things up, the better.

Good night.

Diary 2025-12-06

2025-12-06T05:00:00-05:00

I'm at least keeping pace in terms of designating roots, but I do still need to figure out the right format for the various recombinations I have in mind, and to come up with or generate roots.

Going to go sleep now.

Good night.

Coding 2025-12-05

2025-12-05T05:00:00-05:00

All right. I just finished queueing up more word formation for my future self, so let's look at what I want to do before that.

Aside from anything else, I'd like to rig up a generator for the simpler phonology I converted things to.

But also, I've been considering how I'm processing words at different times, and I think the current data layout is just wrong. Right now, the primary key is the pronunciation, and the "short gloss" disambiguates below that. This is something that kind of makes sense in the context of a dictionary, but when I'm doing evolution, and in particular, selecting particular words to combine in various ways, I think the "short gloss" should be the primary code word, for being generally stable across time, except when I explicitly alter it. The structure for the lexicon is currently a map from pronunciation to a map from "short gloss" to "metadata". I think instead, it should be a map from "short gloss" to a pair of pronciation and "metadata". Then, the drift structure can entirely ignore pronunciation, and also make changing the short gloss optional maybe.

There are two independent strands to this. On the one hand, the lexicon structure has to be updated, the serialization updated, and the deserialization partially updated, the proto lexicon, regenerated, and then the deserialization updated the rest of the way. On the other hand, I'm going to have to update the drift file manually.

The point of all of this is to enable further processing in a more resilient manner than what's currently possible. I want to be able to filter lexicons by metadata, and write input files that concatenate specific words, as well as take products of lexicons. Let's focus on making the required changes, first.

...

It wasn't as simple as I'd hoped. One of the things that Brassica does is handle sound changes that can result in multiple pronunciations. If the short gloss is the primary key, then that means that there could be multiple associated pronunciations. Now, it appears that Brassica properly handles inputs with multiple pronunciations, so I don't need any special logic for applying the sound changes, except that I'd like to store the pronunciations separately and concatenate them only to apply sound changes.

...

All right, I've got things going, and it seems to all be working. I'm not going to try to do any more tonight, but here's what I'm missing that should now be easier:

Word formation via compounding
Paradigm generation

I need to sketch out what the flow for that should look like, but also I can no longer sweep the state of the sund changes file under the rug; it needs to be cleaned up for anything else to be possible. (Also, once it's cleaned up, I should try using it as a template for making that other naming language that I want to use to disambiguate sounds that collide.)

(Also also, there are some checks that I'm simply no longer doing, which I need to find some way to reintroduce.)

In any case, I'm tired.

Good night.

Coding 2025-12-04

2025-12-04T05:00:00-05:00

Okay, I am not ready to set up grammaticalization in my pipeline, so I'm going to make sure I have all of the etymologies set up, and try to lay out what needs to happen.

The missing etymology for cases is a case that I'm calling locative, but it's kind of broader. I'm not totally happy with any etymology I've come up with yet.

From the future: "grassland" would fit reasonably with the metaphor I have for the other cases

So, for the moment, let's consider how I'm planning to handle erosion of grammatical cases.

Split the sound changes into before and after
Combine any individual words that will need it into a "combined lexicon"
Apply a small number of aggressive sound changes to create an "eroded lexicon"
Combine words from the eroded lexicon with the main lexicon, to create a "grammaticalized lexicon"
Apply the rest of the sound changes

Anyway, that's keeping things going well enough for my taste. I'm getting tired, so I'll try to wrap up.

Good night.

Coding 2025-12-03

2025-12-03T05:00:00-05:00

Okay, I've collected my notes on various prepositions, suffixes, and other sundry components of noun phrases. Now, I wasn't doing what I was supposed to, as far as lexical roots, so I've got a bunch of TODOs to fill in, but at least they are all collected in one place, even if I'm absolutely not ready to fill them in right now.

Aside from that, I'm trying to remember what the heck I was doing with Impliciula. Unfortunately, I only got as far as wrangling Tox into not breaking. I'll try to get somewhere with it tomorrow, or later.

For now, I have to get some rest.

Good night.

Weekly Roundup 2025-12-02

2025-12-02T05:00:00-05:00

Wednesday: I switched the data format of my lexicon files.
Thursday: I took that new data format, and updated it further.
Friday: I thought I had everything I needed for representing semantic drift.
Saturday: I realized I needed to tweak the data format for semantic drift.
Sunday: So, I updated the code to process the data, but not the data itself.
Monday: I finally updated the data file.

Next week, I'm going to see how much I can focus on other, unrelated code, while still following the prompts.

Coding 2025-12-01

2025-12-01T05:00:00-05:00

Okay, I updated the data file, ran the code a few times, tweaked it until it seemed to work, and... I have notes.

The current flow is optimized around tweaking a specific gloss, not around adding or removing glosses.
I suspect I'm permitting something bizarre around ordering, but I'm just not going to think too hard about that.
My idea of what constitutes a "short" gloss is suspect.

Probably more things to fix up later, but I got distracted, so I should wrap up now.

Good night.

Coding 2025-11-30

2025-11-30T05:00:00-05:00

It's kind of late again, but I've got this. I made the minimal changes required to rework the semantic drift deserialization. Because it's late again, I haven't tested it, or indeed gotten the data file into a state where it can be tested, but hopefully all that's left now is a relatively mechanical process.

Once I have this working, then I should be following up thus:

Take notes on how I think I'm going to implement grammaticalization.
Clean up all of the Python code I've written to get this working.
Figure out how to clean up and divide up the sound changes, so I can have the sound changes look like they weren't written by a bunch of enthusiastic raccoons with a surprising level of linguistic knowledge.

Anyway, I'll hopefully get to work on that in a day or two. For now, bed is calling.

Good night.

Coding 2025-11-29

2025-11-29T05:00:00-05:00

Okay, it's way too late, I did the brainstorm part of the Conlang Year prompt that I'm currently on, let's hook some stuff up in code so I don't have to recategorize this entry.

...

Crud. The act of actually just looking at the data file revealed that the lazy way of doing things doesn't work here, because it's not accounting for changing the short gloss. This will need a different data layout, new deserialization, and different runtime logic. That's not happening in the next fifteen minutes, so I'm cutting my losses for tonight, and hopefully getting this fixed up tomorrow.

Good night.

Coding 2025-11-28

2025-11-28T05:00:00-05:00

I'm awake! Somehow...

Anyway, I got some roots chosen for the purposes of grammaticalizing later, but also providing other words. Now, let's think about the bare minimum functionality required for representing semantic drift.

First decision I want to lock in: it keys off of the post-sound-change pronunciation, because it makes more sense to me to say "when it's pronounced like this, it means this" than to say "first we change the meaning, then we change the sound". Next, key off the short gloss. Then, have enough data for the full replacement.

Now, if either the pronunciation or the short gloss aren't found, then something has gone wrong, and it needs to bail out with an error.

Anyway, that format is interesting, because it's actually just a lexicon. And something very similar could be implemented for loanwords.

Okay, just bashed out some prototype code that looks really ugly, but I don't want to think about improving it right now. Tomorrow, I'll check whether it even works.

Anyway, that's everything I wanted done, so it's time to wrap up.

Good night.

Coding 2025-11-27

2025-11-27T05:00:00-05:00

Thinking about coding the update logic for individual entries, and here's what I'm thinking: if two "homophones" have the same gloss, then are they really distinct words? As such, I think it makes sense to switch from "array of definitions" to "nested table of definitions".

...

Okay, that was a little fiddly to update. Probably there was a better way, but this worked.

Anyway, the step to actually doing the updates is to figure out what updates look like. Just for expressiveness, I kind of want the ability to add or partially replace stuff, but simply allowing for replacements of the gloss is technically sufficient.

I'll think about that more later.

Back to the actual prompts. I've made some decisions about word boundaries, but there are plenty more to make when I've nailed down more of the grammar.

Also, now that I've fixed up some expressiveness issues in the build system, I should see about breaking the sound change stuff into different eras, which is unfortunately going to lay bare just how much of a hack job the current state of the changes is.

I just peeked ahead, and I have tomorrow's prompt basically already done. If I have time tomorrow, I'll work on filling in the gaps that I've created in the lexicon.

Anyway, I have more to do tonight, and I can't do it while I write this, so I'm going to wrap up for now.

Good night.

Coding 2025-11-26

2025-11-26T05:00:00-05:00

All right, I did the prompt for today, I think, so let's see about updating the lexicon representation.

(Also I submitted a 200 Word RPG. It's a very specific kind of generic, or possibly a generic kind of specific.)

Anyway, let's crack open the tomlkit documentation and see how I want to serialize stuff. I didn't have to think about this before, because the last project where I was doing something like this, I think I was only ever entering data manually, so I only needed code for deserialization.

First up, it kind of bothers me to put the different keys directly at the top level, so I'm going to need to come up with a dummy table to contain them all. So, we start with a TOMLDocument, which maps a constant key to a table, which maps pronunciations to aot, where each table therein has keys to string values, and an array of string values.

...

All right, a little monkeying around with the regeneration command, and now we've got a data format that won't constantly mess up the line endings. It is much longer, in terms of lines per entry, so maybe I want to look into making some kind of condensed version that can be generated from the TOML version.

But the key thing to think about next is, what does it look like to represent changes to an entry?

I'll see about that later, because now it's late.

Good night.

Weekly Roundup 2025-11-25

2025-11-25T05:00:00-05:00

Wednesday: I attempted to reorient myself with the Conlang Year prompts.
Thursday: I continued on trying to nail down sound changes.
Friday: I thought I was done fixing mistakes in the sound changes.
Saturday: I wrote up the phonotactics and drafted the romanization.
Sunday: I considered some improvements I wanted to make to the code side of things.
Monday: I made one of the improvements, but not the other.

Next week, I'm going to try to change the data format for the lexicon, and otherwise keep up with the prompts. Also, I've got a 200 Word RPG draft to look over.

Coding 2025-11-24

2025-11-24T05:00:00-05:00

I updated the build generator so I don't get an exponential explosion of rules, and fixed yet another sound change issue. And another just now. This is annoying and subtle.

Anyway, I haven't yet committed to switching from TSV to TOML, though I'm probably going to. Let's take a look at tomorrow to see what kind of effort I'm dealing with next. Oh, that won't be too much effort to write up. Those are grammatical decisions that I've basically already made.

Oh, also, even though I haven't yet made the changes to my sound change script to handle updating the lexicon, I did at least go through the current lexicon and do a vibe check on what semantic shifts I expect for each word.

I'll hopefully be ready to come back to this in the morning.

Good night.

Coding 2025-11-23

2025-11-23T05:00:00-05:00

All right, I touched up the romanization ideas a little, and now I need to go over how I plan to update definitions. There are two basic questions I need to answer here: how do I do this (what does the code look like?) and when do I do this (where does this fit into the build specification?)

It feels bad to me to construct a lexicon that has outdated glosses, and then a separate one with updated glosses, so I guess I need to add an option to the sound evolution command to take a file of glosses to update. Then I need to add another bunch of functions to my build file generator, and ponder whether I'm doing things wrong in some way. Like, right now, I'm adding new rules for every command-line flag, which is not sustainable; I must be getting something wrong there. After this, I need to consider changing the lexicon format, but let's consider this for a bit. When I stored a filename in a variable, I needed to make sure it was quoted to stop it from expanding into multiple files. So, I can store multiple things in a variable, space-separated. This means I should be able to store the command-line arguments in one variable using shlex, which is a better idea all around anyway. So, that's a solid plan for updating the build file generation. If it's stupid, and it works, then it's not stupid. However, it is about to stop working. So, I'll have to work on that early tomorrow.

Now, let's consider the lexicon format. I had this idea that it would be cool to work with the lexicon in tabular format. It's frankly kind of painful, and I'm currently considering switching it to TOML.

Yeah, at least you wouldn't be storing coordinates in nested key names. What was that?

Well, it worked, so it wasn't stupid.

Anyway, the current format is a tab-separated file alphabetized by the IPA representation of the words, with columns for "short gloss", "long gloss", "part of speech", and "tags". Homophones get multiple entries. Now, it wouldn't be too hard to have a similar set of tables in an array, to more closely match the run-time representation. One of the things I need to consider, with a more flexible format, is whether I want to try fitting derivational tables into the lexicon. The issue there is that it would be harder to make sure the data gets serialized and deserialized properly when applying sound changes, so I'm right now thinking I'd like to consider how the data gets from the file that works as input and output to sound changes, into a nicely-formatted document.

(Let's also consider the possibility that I put all of the gloss changes in a separate file just to get things done with tomorrow, and actually integrate them later.) Anyway, got to sleep on this.

Good night.

Diary 2025-11-22

2025-11-22T05:00:00-05:00

I'd kind of forgotten how bite-sized the Conlang Year stuff is when I'm not flailing around with sound changes for, I think, longer than was intended, so I was able to do a quick writeup on modern consonants, vowels, and phonotactics. There are some details in there that I didn't explain properly, or nail down, but I've got all of the stuff that was in my head written down, so let's go.

Checking ahead, I see that tomorrow is considering romanization. I can basically nail down what I think makes sense off the cuff:

long vowels are simply doubled
palatal approximant is y
postalveolar fricatives are sh and zh
postalveolar affricates are ch and j, but geminate as tch and dj
otherwise gemination doubles the consonant, and specifically the first letter in a digraph
coda nasals are represented as n
other coda consonants are represented by "the same letter"
coda consonants are followed by an apostrophe if the next letter is the same, a vowel, or an approximant
Everything not mentioned just uses the same letter from the phonology, because all that's left is latin letters with straightforward readings.

That will be a system that is easy to type and read, at least for native English speakers. There's definitely room for trying to be "more precise" or whatever, but I don't see a reason for me to do that besides trying to flex. Which... who's going to care?

Anyway, I let it get late, so time to get ready for bed.

Good night.

Coding 2025-11-21

2025-11-21T05:00:00-05:00

Well, there are some weird cheats still hiding in the sound changes, but I've got some of the bugs that were lingering in there earlier sorted out. At this point, I'm ready to move on, having completely lost track of where I'm "supposed to be".

There are a few things I can come back to:

I want to set up a parallel set of sound changes for choosing between proto-language roots that would become homophones.
I haven't figured out the right way to split sound changes into stages in the context of my build system obsession, with the requirement that I also support the "shotgun testing" code I threw together just now.
Something else I forgot about in between thinking about this entry, and writing this entry.

Anyway, yeah, ready to move on, but even more ready to sleep.

Good night.

Coding 2025-11-20

2025-11-20T05:00:00-05:00

All right, I thought I was going to have more time to reflect on the state of the process generally today, but I guess not. Let's have a look at where I am specifically:

I need to do something with coda fricatives. Maybe they can just undergo fortition to their corresponding stop?
Speaking of which, I need to do stuff with coda stops. Basically, they need to assimilate to a following consonant, or get dropped.
Coda nasals go to small capital n or whatever, initial velar nasals do some vowel-influenced stuff.

Maybe I can do these quickly?

...

Okay, everything is done, except that I haven't got the right behavior for certain approximants yet. Once I've figured out what the right behavior is, I can get it implemented. Perhaps I will focus on evolving a related naming language first, because I kind of want that before I come up with many more roots.

For now, though, I desperately need to wind down.

Good night.

Coding 2025-11-19

2025-11-19T05:00:00-05:00

I'm still "behind" on Conlang Year, but I should be done with sound changes soon, and able to start catching up. Actually, looking at the preview for the week, I might be slightly ahead? I might have misinterpreted an entry and confused myself. Either way, I'm done for tonight, so I'll read ahead a little later.

...

Actually, I just read, and honestly, the way I'm currently handling things, it makes the most sense to do what I'm doing, then fill in some nonce forms, then start Week 6 without caring about which specific day I'm on.

Okay, so, what did I do tonight?

Well, I finally got Ninja calling Brassica, fixed a whole bunch of bugs and "features" in my wrapper code, then started learning Brassica as part of getting about halfway through my planned sound changes. My big insight here is that my current proto-language inventory doesn't do anything too impressive with some of these rules; hopefully compounding them shows more promise, but I need to consider how to set up my code to test that.

For now, I want to wrap up because I'm pretty tired.

Good night.

Weekly Roundup 2025-11-18

2025-11-18T05:00:00-05:00

Wednesday: I started blogging about doing Conlang Year.
Thursday: I threw together some code for the Conlang Year stuff.
Friday: I was tired.
Saturday: I decided to rework the stuff that I had for Conlang Year.
Sunday: I played a puzzle game.
Monday: I made some progress on the code.

Next week, I think I'm going to have to delay Conlang Year stuff a little, because, um, I kept on playing 14 Minesweeper Variants. My goal for tomorrow is to read up on Brassica (the sound change applier) and draft some rules by hand over my lunch break.

Coding 2025-11-17

2025-11-17T05:00:00-05:00

So, I made some progress. I got ninja bootstrapped to build itself, and it should work on the evolution rules I wrote, but I haven't tested this. I ended up needing to handle regenerating the proto-language lexicon in not the way I was envisioning, which is fine. I've updated the documentation to reflect the new phonology and syllable structure. I haven't fixed the generation code; I instead just used vibes to translate its output into something that fits with the new structure. I added nonce forms directly to the lexicon to take up the new phonemes in different positions, and I'll figure out what they mean when I feel the inspiration or I really need a new root.

I'm on track to follow with the prompts, although I am getting kind of antsy to explore the grammatical features that got me motivated in the first place. Looks like I'll be moving towards that in a few weeks.

For now, I want to wrap things up, and rest after going through the mental effort of stuff like "Remembering how my wrapper around my wrapper around Google's wrapper around Ninja works".

Good night.

Diary 2025-11-16

2025-11-16T05:00:00-05:00

I ended up doing less than I planned because I got repeatedly distracted by 14 Minesweeper Variants, a game which I really would like to see a Bombe-like of, because man that's a lot of different deduction frameworks to remember. Maybe I'll get stuff done tomorrow, maybe not, we'll see.

For now, I let things go way late and I need to get to bed.

Good night.

Coding 2025-11-15

2025-11-15T05:00:00-05:00

I'm still a bit out of sorts as far as following the prompts goes, but I'm trying. At the very least, I've confirmed that my tossed-together Python code can apply sound changes to a "lexicon". I think this has enough for me to handle the tech side of tracking the sound changes, so my focus now should be mostly on getting a better handle on sound changes. As part of that, I redid my proto-lexicon with slightly more sensible-looking syllable structure. Before I experiment with it, I'm going to have to audit all of my changes, so I'm not going to do anything with it just yet. There's also the fact that I want to introduce a new stop series to make some of the changes I have in mind work, so I definitely shouldn't be rushing this.

So, what needs to be done is:

Inventory the current syllable structures
Redo the janky generation code to be a lot nicer (because I've switched to a simpler syllable structure)
Alter existing entries to fill in gaps, or add new entries

I'll do that when I feel up for it. For now, bed.

Good night.

Diary 2025-11-14

2025-11-14T05:00:00-05:00

I'm trying to keep up with Conlang Year, but I'm extremely tired right now, and I think I'll go lie down early for a bit, rather than push myself to do anything in particular.

Good night.

Coding 2025-11-13

2025-11-13T05:00:00-05:00

It feels like I'm doing the Conlang Year stuff a little out of order, so I may need to take a day at some point to regroup and review the last few days. I have put together a partial plan for sound changes, but a lot of it is riding on doing a bunch of stuff with vowels.

In any case, let's see what I can do when it comes to implementing the code I want.

...

Well, I've got something that looks pretty much like what I think I want, but I want to take things easy tonight, and not debug the no-doubt horrible problems with it just yet.

I think I'll try to wrap up a little early and take things easy, like I said.

Good night.

Coding 2025-11-12

2025-11-12T05:00:00-05:00

The generic enum code has no documentation and I suspect there are some subtle bugs waiting to be teased out, but I decided to just publish the code and stop waffling. I'll cut a release when I find I need it.

In other news, I've been doing Conlang Year prompts, nearly two years after they came out. I have some serious reservations about the current state of my proto-language, but I'll need to get some proper experience with evolving it to know how real the problems are. My intended phonology is Japanese-inspired, and the proto-language phonology is... not that, so I'm going to need a lot of changes to get there.

Right now, I'm just trying to get ready to experiment with sound changes. There might be a bit of a delay, since I'm trying to get automation set up. Right now, I've got a hand-rolled format for the lexicon (as in, I've assigned specific meanings to five columns in a tsv file), and I'm writing code to manipulate it. Step one, write a command to regenerate a lexicon file. This lets me add entries whereever, and have them end up alphabetized. Step two is a little more involved. I need to actually write the code for applying sound changes; my hope is to outsource that to a CLI tool, but I haven't actually tested the tool that I have in mind yet. None of this should be hard, it's just that I spent a bunch of mental energy tonight on stuff that was neither enjoyable nor blog-worthy.

Since I didn't get the code written yet, I at least picked out some areas of the proto-lexicon where I could imagine a sensible sound change. Once the code is up and running, I'll work on generalizing the changes that stuck out for me.

Fingers crossed I get stuff properly implemented tomorrow. For now, I really need to rest.

Good night.

Weekly Roundup 2025-11-11

2025-11-11T05:00:00-05:00

Wednesday: I did some more work on planning repos to live off of GitHub.
Thursday: I did a small amount more planning work, which is all I was up for.
Friday: Still more planning work, and prioritization.
Saturday: Nothing concrete to report, just more of the same.
Sunday: I believe this was when I started actually touching code.
Monday: I got most of the way through one of these repos.

Next week, I'm going to work on documenting this code, and then actually push and publish it. Also, the Other Project should start getting a little more interesting, so maybe I'll finally feel like discussing stuff.

Coding 2025-11-10

2025-11-10T05:00:00-05:00

Okay, I'm pretty close to being ready to publish my work on generic enum stuff. The big thing I haven't looked at at all is the quality of the documentation, and I'm still not quite understanding some code; I'm missing some mutation coverage. The line in question is obviously correct, but I'm having trouble figuring out what goes wrong when the mutant in question is used instead.

I don't know what I'll feel like poking at over the next few days, but right now I need to wind down.

Good night.

Coding 2025-11-09

2025-11-09T05:00:00-05:00

I'm working on setting up a repository, but it's not there yet. I did get a few things nailed down for this, but I also was working on The Other Project, so I'm not getting stuff done here, um, as fast as I could if that's where my focus was landing. Like, I should have something ready in a day or two, but the virtue of these tiny Python libraries is more that it's something that's not too hard to work on, rather than any deep interest. Anyway, it's late and I want to go read.

Good night.

Coding 2025-11-08

2025-11-08T05:00:00-05:00

All right, I'm not up for too much more tonight, but I'm making steady progress on the not-that-interesting-yet thing, and I've nailed down one of the technical decisions for the small Python libraries.

I was pretty out of it this week, so I'm trying to take it easy until I've had more sleep. I hope to have something worth showing off soon, but I'm not going to force anything. Let's wind down...

Good night.

Coding 2025-11-07

2025-11-07T05:00:00-05:00

I'm making decent progress on the other projects, so let's see if I have anything quick to do for the stuff I'm actually writing about.

All right, so. I have a bunch of candidates for code to factor out first. In each case, I want to do some due diligence to figure out whether something already exists. This already replaced some of my sketchy code with an extant library, so it's not out of the question. The stuff that looks most like I'll need to publish something for it includes:

Weird Enum Stuff
As many terminal color scheme hex codes as I feel like
ContextVar helper code
Some funky metaprogramming based on type parameters

I intend to work on this, or at least on something, over the weekend, but right now I want to wind down. Also, I need to think of names for these, and I'm not ready for that right now.

Good night.

Coding 2025-11-06

2025-11-06T05:00:00-05:00

I was very out of it today, so I tried to take things easy and then do some constrained work. On the side of stuff I'm currently talking about, I laid out a set of things I want in each of the small repos I'm planning to write, and picked out a few repos that look like reasonable starting points.

I didn't actually start anything, because my brain has been fried all day. I should sleep.

Good night.

Coding 2025-11-05

2025-11-05T05:00:00-05:00

Time is passing faster than I'd like, and I'm putting in thought to some side projects that I intend to talk about later, but I'd like to get something done tonight.

As such, I just transferred the notes I took last entry to an Obsidian note, where I'll be prioritizing, annotating, and checking them off. I want to leave it there so I can spend some time thinking about that side project, so I'll end this entry now even though there isn't really anything here. Make sure I'm not scrambling later.

Good night.

Weekly Roundup 2025-11-04

2025-11-04T05:00:00-05:00

Wednesday: I confirmed that the code I threw together for base stuff seems to work, and promptly didn't do anything with it.
Thursday: I looked over the coverage misses in MOTR, and tried to figure out what to do about them.
Friday: I drilled down into the design of the saltate plugin, and rediscovered the various problems with it that I've just kind of been avoiding.
Saturday: I ended up with a new project idea, not fully developed.
Sunday: I figured out that some of the work I had in mind for that project was unneeded, which therefore meant I was closer to completion than I thought.
Monday: I eliminated a bit more work, which meant moving onto the next phase: factoring out code from my hobby projects into a bunch of tiny, inscrutable libraries.

Next week, I need to nail down exactly which libraries I'm writing, their minimum supported Python version, and any enhancements I want to make to them.

Coding 2025-11-03

2025-11-03T05:00:00-05:00

I'm right now very slightly leaning towards switching to Codeberg. At first, I was looking into what it would take to migrate the one repository I think I care about, and then I kind of ended up going, like, do I actually care? The one repository is a fork of flake8-html, that redoes some of the output. Now, I'm not totally migrated from flake8 to ruff in the projects where I've even tried to switch, but in those projects, ruff is pulling the vast majority of the weight.

Rest assured, I didn't merely come up with reasons to not do work; I've got a Codeberg account now, thereby opening up new avenues of procrastination. More seriously, here's how things look for projects I want to put up there:

MOTR (split into three)
Impliciula
The base project
Some of the code I wrote for various projects meant to track tabletop game stuff; this would be the support code, and nothing system-specific. For example, I've got some code for Stars Without Number; I would, rather than anything system-specific, put up the code for mixing enum and attrs classes (actually, I'd use the version from the base project), to create generic enum types, to allow for empty final enums, to convert between different styles of name, the "generate BEM HTML from specially crafted Python objects" code (once I sort out some Jinja misuse), the colorscheme handling, the dice pool code, the funky logging code (unless I just switch to using an actual logger), and the contextvar contextmanager code that I always seem to be writing.

Anyway, I wanted to get more done, but I think I'm still recovering from that night of mostly not sleeping. Nothing else I want to discuss right now.

Good night.

Coding 2025-11-02

2025-11-02T04:00:00-05:00

I took another look at the code that I'm currently installing from my own GitHub repos, and it looks like I should be able to stop doing that for two of them, which can only be an improvement. The remaining one, I think I'm kind of stuck with, but oh well.

I wasn't really focused on this today, so I am going to take it easy for tonight, and tomorrow, I'll either finish this train of thought, or I'll work on the base stuff wiki. Maybe both. For now, I need to wind down.

Good night.

Coding 2025-11-01

2025-11-01T04:00:00-04:00

Earlier, I did some investigation of the Mypy plugins I wrote for MOTR, and rediscovered some improvements that I want. After that, I had a terrible night, and today... You know how I sometimes complain about having had a really bad day? So much worse.

And the night that led to that day, it gave me some thoughts (instead of sleep, which is the big issue), and now I'm going to derive some plans from those thoughts. The main thing I want to do is get set up on a code forge besides GitHub, and migrate my forks from GitHub to that forge, and start publishing my various hobby projects on it. Alternatively, I could look into simply doing something equivalent to hg serve...

All right, quick thoughts before I wrap up. My goal here is to document and publish the process and ideas behind the projects I'm most passionate about. For the purpose I have in mind, self-hosting doesn't quite make sense, so I need to pick a forge or hosting service, although I ultimately don't need too many features.

I think that gives me enough to go on once I've had some actual sleep. Which I should work on.

Good night.

Coding 2025-10-31

2025-10-31T04:00:00-04:00

I thought I was going to get more done today, but whatever. I did a deep dive on the saltate plugin in MOTR, and realized that it's actually kind of broken in ways that haven't mattered to me. If I can fix this, the way to fix it is to find a set of type arguments to the origin of the instance type that cause it to unify with the target type. (I wrote a comment to this effect, but I'd forgotten what I meant until this exact moment.)

My plan here is to copy this code into a new project, touch it up, and publish it.

Quick thoughts on how to implement what I was saying... I guess I need to confirm that I have an Instance, and create a version with synthetic type variables. Can I do that? Seems like a pain, but it looks like I can do stuff with TypeVarType. At that point, it might be possible to pull code out of unify_generic_callable.

It's not worth thinking about this any further tonight, but I can play with this in a day or two.

Good night.

Coding 2025-10-30

2025-10-30T04:00:00-04:00

I didn't realize how rough of a day I was having until a few hours ago. I tried to address the causes as best I can, but I still need at least one night's sleep to handle it.

With just a little time to work on hobby stuff after I cooled off, I decided to see what kinds of fixes MOTR needs as a result of not looking at it for a while. I ended up needing to update one test case, and delete a file that my NAS really wanted to keep around for some reason.

With that out of the way, let's look at the coverage misses:

There are a few lines in the saltate plugin that aren't hit because I would need to break some assumptions of the plugin code to hit them. Probably the strongest candidate for adding a no-cover pragma and not looking back.
The coverage gaps in the type_func (dependent mapping) plugin are actually kind of a bug that I should fix, but I don't want to right now. Since the code should eventually change in the exact place that's not being covered currently, it does make sense to keep checking this.
The cmd module is missing coverage of EnvVar.concatenate. The natural way to get coverage is to pass an Invocation to Invocation.append such that their env attributes have overlapping keys. This may require going through InvocationRelations.append. This is called from InvocationEnvironment.append, which is called from... places.
The command module is missing coverage of Implicit artifacts, as well as various error cases in a helper function that's ultimately called by Command.matrix and Command.fill_in_module.
invocation_environment is missing a line that can only be hit if InvocationEnvironment.invocation_relations contains an Executable as a relation, and we get the facts. I don't remember what some of this means.
I'm running out of steam, so I'll look over runner and matrix later.
That said, conftest is also a case of "I'm not feeling worried about putting a pragma on this line, I just haven't done it yet".

For now, I really need to get some rest.

Good night.

Coding 2025-10-29

2025-10-29T04:00:00-04:00

All right, here's how things are: the prototype code for generating tables in Wikidot syntax works fine. As does the base conversion code. The way I want to actually use this is to set aside pages just for tables, and use the include directive to put them into the actual "article" pages. This way I can edit the table without worrying about accidentally deleting part of the page, for example.

I want to wait a little before putting this stuff together; maybe work on it over the weekend. Until then, I'll try to get back to other projects. Perhaps make progress on Impliciula or remember what I was doing to MOTR. For now, I need to wind down.

Good night.

Weekly Roundup 2025-10-28

2025-10-28T04:00:00-04:00

Wednesday: I cleaned up some of the base-processing code, and found an article to read.
Thursday: I planned out representing different kinds of complex integer.
Friday: I read enough of the article to not be totally sure that I want to implement it, but there's still some interesting stuff in there.
Saturday: I poked further at the article, and got most of complex base stuff implemented.
Sunday: I implemented a bunch of random code, and then remembered that I intend to actually share this stuff, so it needs some manner of code to generate tables and charts.
Monday: I got distracted and refactored the code a bunch.

Next week, since I prototyped the table generation code before writing the roundup, I actually should be ready to start implementing stuff for the wiki.

Coding 2025-10-27

2025-10-27T04:00:00-04:00

Rather than address the fact that all of this base stuff needs to go on a website, I futzed around with the class hierarchy so that I can eventually support rational and complex bases. There's also the proper jumping-off point for algebraic bases, but I'm still struggling with how to handle them in a generic fashion.

Although I did a bunch of refactoring, I don't have much more to say about what I did, so I'll just note that I'm going to try to either focus on the table stuff, or switch projects for a bit.

Going to try to wind down now.

Good night.

Coding 2025-10-26

2025-10-26T04:00:00-04:00

My focus has been drifting from the base stuff a little, but I did still get some stuff done there. I added boolean conversion (which I'd forgotten about before), and handwrote the enums for the balanced complex bases. I just filled in a bunch of other stuff, including writing validators. At this point, the ways forward (aside from polishing existing code) are to figure out what needs to happen to deal with base representations of complex numbers, and to implement more tests and procedures.

Actually, since I brought that up, I just wrote a one-liner for determining whether a number is Harshad, and then realized I need to special-case when the digit sum is zero. And when the number is zero.

Anyway, thinking about this reminded me of one other thing I need to work on, and that's the table-generating code, now that I have enough code written to populate new tables, I just haven't bothered to toss together the generators. That's something I should try to do tomorrow, in between the other things I'm working on, which I'm going to keep under wraps for now. For the moment, I'd better get to bed.

Good night.

Coding 2025-10-25

2025-10-25T04:00:00-04:00

All right. As I get a better grasp on what this article is saying, I'm further from sure that I can directly use it. Basically, it defines two infinite families of algebraic numbers that are suitable as bases, but these families are not exhaustive. For instance, if I'm understanding the families correctly, neither of them contain the plastic ratio, which is suitable. However, messing around with the families did reveal something interesting.

$x^2 - 3x + 1 = 0$ has greater solution $\frac{3 + \sqrt{5}}{2}$, which is between 2 and 3, so it uses ternary digits. The carrying looks a little confusing, until you see that a value of three or above in a place carries both up and down at the same time. Unless I'm missing something, there's no maximal version of this base, because there's no way to construct the equivalent of two consecutive ones in phinary. My intuition is that the base isn't doesn't have a maximal version if the negative coefficients are all consecutive, and all of them are have magnitude equal to the highest digit in the base, except for the last one, which has magnitude equal to one more.

Anyway, I don't have further ideas of what to do with all of this just yet. While I was typing this up, I tried to implement complex integers. There's a little work to do to get the display of them nice, but they should be ready to go for purposes of doing arithmetic to them, which means I can think about moving IntegerBase code into a base class and extending it to work with complex bases. I really should see about writing proper validators and invariant checkers, though...

Oh well, none of that is happening right now, because it's late and I'm going to get ready for bed now.

Good night.

Coding 2025-10-24

2025-10-24T04:00:00-04:00

All right, I'm not feeling great today, but I managed to skim that article on algebraic bases, and I'm not sure how much I'll be willing to apply from it. If I'm understanding the article correctly, it establishes by an inductive argument that bases of a particular form have terminating integer representations. However, it appears that, if I base things off of the constructions in the article, then actually obtaining those terminating constructions is going to be linear in the magnitude of the integer. I suppose I could get that to work "better" by caching the conversions, but I'd feel better if I could figure out, say, both incrementing and doubling to construct digit sequences. That way I could ask for, say, one million in phinary, and it would take something like thirty "operations" rather than a million.

So, for sure, it's good to have a restriction on the algebraic bases I want to consider, but I think I'm going to need to do some more research or experimentation. I don't know if this is totally off base (or if I maybe already said something like this before), but the conversion between "base representation and polynomial function of lower degree" feels analogous to the Fourier transform, or inverse Fourier transform, according to my intuition. Just got to... invert an infinite-dimensional matrix? To produce a vector that is sparse in non-zero entries, drawn from a subset of the integers? This idea is not baked all the way.

Anyway, it's late, and I need to sleep.

Good night.

Coding 2025-10-23

2025-10-23T04:00:00-04:00

All right, I didn't have much time to post today, so I'm going to try to lay out some work I need to do on the base stuff.

For now, I'm not considering algebraic numbers. Just integers (built in) and complex integers. For complex integers of either kind, I want:

Add instances of a type to each other.
Subtract instances of a type from each other.
Add and subtract integers.
Divide instances by each other, which requires:
Take complex conjugate of instances
Multiply instances by each other
Convert real-valued instances to ints
Divide instances by ints
Also, checking divisibility of instances by each other, so, taking the modulus I guess?

I'm probably also going to want to figure out stuff like "hey, is this instance an exact power of another instance", which is going to be... interesting. (Wait, multiply by conjugate and check power; there might be some false positives from this, not sure.)

Anyway, it's late and the melatonin is kicking in, so I'm not going to try to do any of this now, but this all feels like something I could throw together with an abstract base class and a bit of elbow grease.

In any case, I need to get ready for bed now.

Good night.

Coding 2025-10-22

2025-10-22T04:00:00-04:00

I'm not planning to do too much today (I got seriously messed up by the sun), but I can explain more of what I did yesterday, and what I plan to do later.

Yesterday, I realized that having digit sequences as instances of base-specific classes didn't make sense to me, so I created a bunch of generic digit-sequence classes, which implemented stuff like digit sums, and will also implement stuff like palindromes. With this, I was able to convert the integer base class from a base class that must be subclassed, to just a class that must be instantiated. By messing with this, I was able to create an implementation of digital roots that is probably robust enough to go on a currently-hypothetical base class of bases. This robustness is not guaranteed by the fact that it seems to work on optionally-balanced integer bases, because in bases like maximal phinary, some small numbers are less than their own digit sum. I believe this still converges, but I've neither put in the work to establish that, nor checked whether anyone else has. And the current implementation will just OoM if I manage to come up with a sufficiently pathological base, so I guess I just shouldn't do that.

Anyway, I stumbled across a paper from thirty years ago that purports to explain which polynomials have roots that are nicely-behaved as a base, which I'm grateful for, since I've experimented with various polynomials, metallic means, and Pisot numbers, and some of them are just not well-behaved. From looking over the beginning of this paper, it looks to be saying that particular classes of polynomials will be well-behaved and usable, irrespective of degree. So that's actually very exciting; it's just too bad that sometimes it uses words that I have literally never heard before.

Anyway, let's see what I've got to work on as I feel like (which right now, I do not):

Read the article, sketch out ideas.
Clean up existing code (document assumptions and potential points of failure, add validation logic)
Extend the code with new calculations.
Implement conversion logic for new bases (rational with number of digits equal to numerator, Gaussian, Eisenstein, algebraic)

Okay, that's about the state of things, at least, things that I feel like mentioning. I'm going to wrap things up early and watch funny videos.

Good night.

Weekly Roundup 2025-10-21

2025-10-21T04:00:00-04:00

Wednesday: I put out a bunch of ideas for the Weird Base Stuff project.
Thursday: I considered more ideas, some of them kind of... not good.
Friday: I worked with a few more ideas, but got kind of stuck on the fact that I don't fully grasp the mechanics or broader implications of the Abel-Ruffini theorem, no matter how many explainer videos I watch.
Saturday: Although I found out about more weird base systems, I did at least try to write some prototypes.
Sunday: I made a little progress on Impliciula, and a little progress on this other project.
Monday: I described roughly how I wanted the prototype to look in some areas, and that was the push I needed to work on it.

Next week, I'm going to try to refine and expand on these prototypes. (I've already reversed some bad decisions that were over-complicating things.)

Coding 2025-10-20

2025-10-20T04:00:00-04:00

I'm still feeling a little bit iffy today, so I'm once again going to try to wrap things up quickly and put down the screen, before I end up feeling worse.

Here's the deal with the base stuff that I'm currently considering: there is some form of class factory that takes as parameters the base, and whether it should be balanced. The idea is that instances of the class will have a sign attribute and one or more digits, with no base point. There will be a constructor classmethod that takes an integer argument and processes it into an instance, and the first thing it needs to determine is whether to set the sign attribute. Three things need to be true for it to be necessary to set the sign attribute:

The input number is negative.
The base is positive.
The base is not positive.

And I just meant to get some stuff planned, but I've just thrown together a very rough prototype. A lot of the validation logic is either not written, or temporarily commented out because it was breaking things, but I believe there's enough functionality in this now to think about questions like "no, really, what interface to this actually makes sense?"

Really not ready to consider that right now, though, so let's wrap this up and switch to paper or e-ink or something for the rest of the night.

Good night.

Coding 2025-10-19

2025-10-19T04:00:00-04:00

I got really exhausted today, and I feel like getting food after that kind of reset me. I made some progress on Impliciula, but any further progress is, I think, going to require some feedback. One positive thing is that in the Mypy release (over a month ago?) they added a feature that is specifically useful for the weird stuff I'm doing in this project, so I got to remove like twenty type-ignore comments.

Aside from that, I was thinking more about the base representation project I got enough together to generate the data required for various kinds of integer base, and it should be ready for me to bash together a prototype that can actually handle the conversions, but I need to think more about what I want the actual interface to look like. Things to ponder include "only unbalanced positive bases need to represent sign separately, and phrased like that, it sounds like a corner case, but really, those are the standard bases".

Anyway, I don't want to agonize over trying to get more meat into this post, especially not at this time of night, so let's wrap it up.

Good night.

Coding 2025-10-18

2025-10-18T04:00:00-04:00

I keep on finding out about more base systems to try out, but I am making efforts to put together prototypes. I've created a helper method to generate basic enums of Glyph objects, and, in the process, found a problem with my attrs + enum wrapper that I think was easier to fix with aspectlib than it would have been with unittest.mock, so I guess switching was the right call.

I'm a little unfocused right now (in part because I had more sickness-adjacent things happening, which, uugh). I did just throw together a stub of Gaussian integers, but what I actually need to be working on is creating the various integer bases that I basically know how to put together.

But that comes after I get some dang rest.

Good night.

Coding 2025-10-17

2025-10-17T04:00:00-04:00

Let's see what I'm thinking... The bases I'm considering have two basic options for digit sets. Almost everything works with the "standard" option of starting with 0, and next most common is the "standard" balanced option, which just requires an odd number of digits. I also came up with two balanced sets of digits for complex numbers, and these sets of digits significantly constrain the bases that they can be used for.

Something to look into: some complex bases don't work; why? Is it having a positive real part? Does doing a "standard" balanced base fix the issue for complex bases that normally wouldn't work?

In any case, I'm getting some ideas for how to represent linear transformations more compactly. Basically...

For a one-element vector, there's nothing that needs to be done.
For vectors representing complex integers, it suffices to simply represent the complex factors as a fixed vector, and similarly-ish for the shear transformation. (Actually, because the real numbers have to be unchanged, the shear transformation is technically just a scaling factor for the non-real coordinate.)
For vectors representing powers of (ideally) a Pisot number, all we actually need to know is the coefficients of the minimal polynomial.

Given these representations, we can then derive the digits required for a base:

The funky balanced complex bases should just be a bunch of hardcoded choices.
Generic complex bases have a digit cardinality equal to the square of the magnitude.
Real integer bases have a digit cardinality equal to the absolute value of the base.
Working things out for the algebraic numbers seems potentially trickier. The questions I need to ponder are, at what degree does it become infeasible to calculate the ceiling of the largest root, and is comparing a sum of powers of that root against that root morally the same as finding that root, or can the relevant symbolic manipulation be pushed to quintics or beyond?

I assume the answers to those questions are actually either obvious, or have straightforward writeups if I knew what to look for.

I mean, I can of course hardcode this for some things, and it doesn't look horribly infeasible to automate some of the calculations for quadratics, but I want an actual sense here of "what will work" vs "what will horribly suck" vs "haha, no".

Anyway, I think I'm going to need to prototype this some before I can articulate what I need to know, well enough to understand the answers. Maybe I'll have time for that in the next few days, maybe not. Either way, better sleep.

Good night.

Coding 2025-10-16

2025-10-16T04:00:00-04:00

I'm poking at the "glyph" implementation; my current plan is to represent negation with an overbar, and complex conjugate with a superscript asterisk. For i, where both do the same thing, I think I'll use the overbar.

With that decided, I'm next pondering the "vector and matrix" concept. Basically, I want to represent various kinds of, I believe the term I'm looking for is algebraic integer, using small vectors of integers (and sometimes rationals), which can be scaled up or down by the base by multiplying them by matrices of integers (and sometimes rationals). Furthermore, in the complex case, some calculations will require an inner product of the vector of interest, and a base-dependent vector. All of that's as may be, from a theoretical perspective, but I'm not sure what kind of implementation would be useful. One problem is my urge to make everything general, when none of the bases I'm currently considering would require vectors with more than three elements. One thought I have here is that it's not the specific matrix representations I care about, it's the linear transformations for shrink, grow, shear, etc. These transformations are specific to a particular base (even if the vector is in the same form for different bases, like Gaussian or Eisenstein integers).

As I think about this, I find myself thinking about creating classes, and composition versus inheritance. Perhaps I can sketch out the relationships between the different bases, and figure out what would best represent that. (I think tossing everything together into inheritance would probably work, but I've got a good feeling about defining new class keyword arguments.) But I can't trust myself to get this right unless I actually have some notes, instead of just keeping everything in my head. I think I'll publish this and grab my notebook and try to get stuff together.

But for sure, I need to get ready for bed.

Good night.

Coding 2025-10-15

2025-10-15T04:00:00-04:00

I'm on a bit of a time crunch because I took a melatonin ten minutes ago, so let's see if I can get some thoughts down really quickly.

If you're converting an integer to a natural number base, you can either start generating digits from the least significant digit (modular arithmetic and division), or the most significant digit (divide it until it is less than the base, then determine the current digit, then multiply). For algebraic bases, the most-significant-digit method seems easier to implement, because you're usually not going to know which digit is the least significant. For complex bases, the least-significant-digit method seems easier to implement, because it seems that complex bases have fractal boundaries between digits, instead of, like, points. I think using least-significant-digit for real integer bases seems somewhat more elegant, just because it's all remainder-less division, rather than dividing to create a fraction, then multiplying it back out.

I've worked out a few bits of information to account for:

When a maximal version of a base exists (which will be algebraic, and therefore use msd), the "width" of the interval to divide to fit in is one plus the base, rather than the base.
Positive non-balanced bases use the half-open interval from 0 to the width.
Balanced real bases (because complex bases don't use the width) use the open interval from minus half the width, to half the width.
Negative non-balanced bases use a few easily-calculated functions of the base which I don't have my notes for in front of me.
Numbers in real integer bases may be represented by a one-element vector.
Numbers in complex integer bases may be represented by a two-element vector.
Numbers in real algebraic bases may be represented by a vector with the same number of elements as the degree of the polynomial.
The complex vector can be sheared into the real numbers and then worked with modulo the number of digits in the base.
The algebraic vectors can probably be manipulated to determine exactly whether specific thresholds are crossed, without relying on floating point.

And I'm also trying to hammer out some ideas for dealing with stuff like "a bunch of these bases have the same digits, but the vector representations are completely different, but in some way related to the specific digits".

I don't know exactly when I'll have time to really thoroughly work on this, but at least I managed to rewrite my code to patch attrs class magic methods so they can be used as an enum base class, to use aspectlib instead of unittest.mock. Like, that's probably better, right?

Anyway, I'm loopy now, so let's wrap this up and get to bed.

Good night.

Weekly Roundup 2025-10-14

2025-10-14T04:00:00-04:00

Wednesday: I made a little progress on Impliciula.
Thursday: I pushed myself a little harder than I could actually handle.
Friday: I discussed an idea for yet another new project.
Saturday: I put together a site for that project, and didn't accomplish much else.
Sunday: I filled in a table manually for that project, and decided not to keep doing that.
Monday: I started pondering how to automate table generation and base conversion.

Next week, I'll hopefully get something together on some project or other.

Coding 2025-10-13

2025-10-13T04:00:00-04:00

Okay, let's see how today is going. I was kind of out of it for some reason.

After last night, I want to figure out whether it's feasible to automate some of the work in doing weird base-dependant stuff. So far, I've got a sketch of table generation, but I can't really use that unless I figure out how to automate base representation. This isn't too hard for natural bases, but the whole point of this project is to mess around with stuff that's un-natural.

Let's see. I want to work with real algebraic bases, real integer bases, complex integer bases, and balanced versions of both.

Complex bases can be an issue due to moderately strange carries, algebraic bases can be an issue due to really strange carries and the minimal/maximal distinction (which totally changes the behavior of digital roots, at least in base phi), and anything long-division-like runs into some snags both in balanced bases, and I think in any base below phi. Although I'm mildly curious how other Pisot number bases work, I think I have to dial my initial expectations down to designing and using a toolbox, rather than getting it all to work perfectly for arbitrary bases. That said...

If a base is a root of a polynomial, it seems to me that representing a specific number should involve a Laurent polynomial that is a product of the base polynomial, provided that, if the target number is added to both sides, all of the coefficients are from the set of allowed digits. Or something like that. I'm not sure whether bringing in theory like this will make the implementation any easier, so I may just need to consider formal manipulations.

The naive implementation would be to repeatedly add one to the representation, but as I think about it, I wonder whether constructing the base representation from binary, by multiplying by two and adding one as needed, would be workable. The basic thing missing there for algebraic bases is how to canonicalize the representation. The multiplication and adding would potentially produce arbitrary digits, in addition to disallowed digit sequences. Waiting until the end to canonicalize would simply defer all of the actual work to the canonicalization step, so let's suppose we canonicalize early and canonicalize often. The problem I have visualizing this is that algebraic bases don't seem to have an obvious direction to work in, since triggering a carry will change less significant digits as well as more. Another possibility would be...

So, until the target number differs from a product of the base by one of the allowed digits, multiply the target number by the base. Then, do long division from the least significant digit, like something with p-adics. Getting this to work requires something like a vector representation of arbitrary digit sequences in the chosen base, along with matrix representations of multiplication and division by the base. I haven't done the legwork, so I'm going to just kind of hope that all of the bases I'm interested in have a diagonalizable matrix, although I'm not actually sure how helpful that would be; it just seems probably helpful.

I don't have a sense for how this would go in code, so I'm going to have to experiment with it by hand for a bit. (Maybe it needs normal long division, if I can somehow tell that there's no more division to do.)

Anyway, I want to wrap up and wind down. Maybe later I'll have some more ideas for this.

Good night.

Diary 2025-10-12

2025-10-12T04:00:00-04:00

I took things easy today, but I did make a little progress on a wiki. I didn't do things in quite the way I was thinking yesterday, but I did lay out some tables. That was obnoxious enough that I think I need to look into writing code to generate them.

Anyway, I let things go late again, so I'm going to wrap up now.

Good night.

Diary 2025-10-11

2025-10-11T04:00:00-04:00

Okay, I decided to take things easy today, so I have nothing to write about. I did start up a wiki for that project I mentioned, but it's private and doesn't have any actual content yet. I think what I'm going to need to do for this wiki projects I have lying around is, go over what I have, and focus on identifying problems before articulating solutions. Shift mental gears less, stuff like that. Not doing that right now, though.

I'd better wrap things up, because it's already late.

Good night.

Diary 2025-10-10

2025-10-10T04:00:00-04:00

Going to dash off a quick entry, because my focus/energy is still kind of... not what I want it to be.

One thing that I've been pondering in the last week is the idea of a recreational math project themed around bases. Like, you know how there's base-independent recreational math, like amicable numbers or perfect numbers, and there's base-dependent stuff, like palindromes, digit sums, that kind of thing? Well, there are also efforts to develop various "weird" bases, and I wondered what it would look like to analyze the base-dependent properties and concepts in the context of "weird" bases.

Right now, I'm trying to figure out how I'd want to present it. I guess I'm currently trying to figure out what would have an engaging presentation, but not be too onerous to edit. Is it going to be a wiki? It might be a wiki.

Anyway, I should get to bed now, and continue this line of thought later.

Good night.

Diary 2025-10-09

2025-10-09T04:00:00-04:00

Oh boy. I ramped things up today, tried to get a bit more done than I did while I was sick, and now I'm just, so tired. Feet sore. Back hurting or something. Eyes operating at somewhat diminished capacity.

I'm just going to get ready for bed as soon as possible.

Good night.

Coding 2025-10-08

2025-10-08T04:00:00-04:00

I decided the simple solution was good enough, and named the Sequence attributes initial_comp, binding, and final_comp, instead of *mumble mumble mumble*.

I'm about ready to poke at the type attributes, but I'm really leery of trying to "just one more thing" this work right now.

I don't have anything else in mind to do right now, so I'm just going to take things easy.

Good night.

Weekly Roundup 2025-10-07

2025-10-07T04:00:00-04:00

Wednesday: I recovered a little, but not much.
Thursday: Again.
Friday: Again.
Saturday: I played Kaizen and had some thoughts on it.
Sunday: I played Strange Jigsaws and had some thoughts on it.
Monday: I started poking at Impliciula again.

Next week, I'm going to try to keep doing stuff, even if I do seem to have some lingering illness still.

Coding 2025-10-06

2025-10-06T04:00:00-04:00

I'm methodically working through the different node types, and seeing if there are any changes I can make in the name of clarity. So far, I've made some changes to the function nodes (abstraction and application) and I know I need to change the attribute names in the sequence node somehow, but I don't know to what.

What we have in the sequence node is an initial computation, which should return a value, which is substituted for a variable in the subsequent computation. I have pretty bad names for these, and I'm trying to come up with better ones.

Right now, though, it's late, and I should get ready for bed.

Good night.

Diary 2025-10-05

2025-10-05T04:00:00-04:00

I kind of zoned out all day, appreciating how it mostly doesn't hurt to breathe.

While I zoned out, I bought and beat Strange Jigsaws. Fun little afternoon diversion, though I did get randomly stuck on one of the puzzles, I want to say near the midpoint, but I'm not really sure. Strange Jigsaws is accurately named, so if it sounds interesting, go for it.

Anyway, it's late and I should keep getting rest.

Good night.

Diary 2025-10-04

2025-10-04T04:00:00-04:00

Kaizen Kaizen Kaizen.

Yeah, we all thought that wasn't ever going to happen again. But I finally took the new not-Zachtronics game for a spin, and just-one-more-leveled my way through a good portion of the early game.

My general impressions thus far are that this is probably the most approachable zachlike I've tried (synthesis, not coding; no transmutation; square grid; tracks don't turn; only have to handle one cycle (which also allows for a live preview of what happens); there's a limited and global bank of instruction slots; welding happens without an explicit instruction, but only where the player specifies; no player intervention is needed to output the product; I haven't needed to make use of a wildly specialized control scheme to play it on the Deck, and it should work equally well with just mouse), although I can guess at some of the tools that could show up if, as I assume, the back half of the game introduces some new tools. (To be clear, "approachable" is not a pejorative. If I want to get seriously challenged so I can show off my leet zachlike skillz, TIS-100 is still there for me. Waiting. Watching. Mocking.) Anyway, the core puzzle stuff feels nice, but the bundled solitaire game (it really is a not-Zachtronics game!) is beyond my comprehension and patience right now.

And those are my initial impressions from playing the early stuff while mostly not sick. It feels like it could be a solid introduction to the genre, which seems backwards, but oh well.

Anyway, it's late now, and I'm going to wrap up.

Good night.

Diary 2025-10-03

2025-10-03T04:00:00-04:00

My condition continues to improve, though I'm pretty sure I haven't gotten all the way back to my normal capabilities, even discounting that my throat is still messed up. I'd like to get back into the swing of things, but I think that won't be possible or even truly desirable, until I actually get all the way better.

Not sure what else I could say for now. Better wrap up.

Good night.

Diary 2025-10-02

2025-10-02T04:00:00-04:00

All right, I feel things clearing up. Not all the way recovered, for sure, but definitely improving.

The quality of stuff to write about, however, is not. I'm going to wrap things up and try to get slightly better sleep than in the past week.

Good night.

Diary 2025-10-01

2025-10-01T04:00:00-04:00

I sort of assumed I'd get something done to write about today, and I honestly cannot account for how I thought that. I think my health did somewhat improve over the course of the day, going from slowly eating breakfast because I figured I'd regret not eating breakfast, to eating as much dinner as I could manage, but at no point did I attain a level of health that I would describe as "good".

I'm going to go lie down, which, wow, what a change of pace.

Good night.

Weekly Roundup 2025-09-30

2025-09-30T04:00:00-04:00

Wednesday: I laid out some ideas about making calendar calculations more straightforward, at least to implement in code.
Thursday: I complained about, roughly speaking, the economy.
Friday: I tried to put together plans for later.
Saturday: I was sick.
Sunday: I carried out those plans by rambling about hypothetical calendar systems for non-elliptical orbits.
Monday: I was still sick.

Next week, we'll see, since I'm currently still sick.

Diary 2025-09-29

2025-09-29T04:00:00-04:00

I'm now less sick, but I'm definitely experiencing symptoms still.

I don't remember doing anything today that I want to talk about.

My head still feels funny.

I'm going to wind down now.

Good night.

Tabletop Worldbuilding 2025-09-28

2025-09-28T04:00:00-04:00

(This is getting away from the category name a bit, but it's in theme with the other entries, so... I'm just going to put off thinking about how to "fix" this until I forget that it's a thing.)

Okay, I'm doing not-great, because it hurts to talk, but I'm still thinking about stuff, so let's get into it.

I basically understand how to do year-related calculations for an Earth-like planet, in particular, one with a low orbital eccentricity, and nothing else in the vicinity strongly interfering with its orbit. Anomalistic year combines with apsidal precession, giving mean sidereal year, which combines with axial precession to give the familiar tropical year.

Suppose we have a planet with extreme orbital eccentricity. In that case, the anomalistic year would be of primary concern (because part of the planet would be much warmer at pericenter), with the tropical year being secondary or irrelevant. If the apsidal and axial precessions did not cancel, then there would also be a long-term cycle of libration of the warmest latitude at pericenter, oscillating between the tropics. Things get confusing if the seasonal cycle is still relevant, because being near pericenter speeds up the sidereal and tropical years.

What if the orbit weren't a straightforward ellipse? What about tadpole orbits and horseshoe orbits? Fair warning, I'm shooting from the hip here, because my attention span is not at my best right now. The first thing I note is that "period in the rotating reference frame" takes the place of the anomalistic year, and "rotation of the reference frame" takes the place of apsidal precession. For planets of comparable mass in a horseshoe orbit, this should be all that's needed for a decent mean sidereal year, which can then plug into everything else. This is because the inner and outer orbits will have orbital periods within, say, two percent of each other, by Kepler's laws. Although, given that I'm imagining someone on this hypothetical planet caring enough to get a calendar right, at some point someone will realize that the day length changes by like five seconds every thirty years or so, and this will bother them.

"Yes, yes, the other planet grew to the size of a second moon in the sky, then receded, but I'm more concerned about how all of my clocks are off by the same amount, and more every day. Wait, what if my clocks are right and the day is wrong?"

You're poised to corner the market on a very specific sub-genre of science fiction.

Thank you.

Anyway, now I'm trying to figure out what it would look like to have a very precise calendar with comparable masses, or a doesn't-get-extreme-anomaly calendar with different masses. The first idea that comes to mind for me is to maintain two tropical calendars, one for each range of orbital distance, and somehow interpolate between. them. However, since the switchover doesn't change the current season, there could be a "unified" calendar with a system of intercalation. This would then leave the problem of how to determine when to switch, and how to keep the year right over long periods of time.

However, I'm about done thinking about this for tonight, so I'll just let that question hang, and wrap things up for now.

Good night.

Diary 2025-09-27

2025-09-27T04:00:00-04:00

My throat has felt awful all day, and I've been dizzy and had trouble focusing, so I'm not even going to try to say anything interesting. This weekend I'm going to be focusing on physical paper, when I'm not lying down waiting for my body to heal itself. Uugh.

No point in stretching this out.

Good night.

Diary 2025-09-26

2025-09-26T04:00:00-04:00

Another day, another tenuous alignment between the things I'm "supposed to" be thinking about, the things that interest me, and actually putting anything into practice. Let's lay stuff out so I can try to have a plan later:

Secret project that will never be elaborated upon here.
Discussing Impliciula so I can refactor it a little.
Various other Python projects, including MOTR and audio synthesis.
The calendar stuff; I think what I need there is to focus on generating more tangible representations of this stuff.
A confusing pile of writing projects. Perhaps I should just write something and then figure out what it's "for" afterwards.

That's everything that's coming to mind right now. I'll think it over in the morning. Right now, I want to get to sleep, so I can hopefully avoid dealing with stuff like how messed up the skin on my hands is right now.

Good night.

Diary 2025-09-25

2025-09-25T04:00:00-04:00

I was wiped out like an hour ago, so I don't think I'm going to come up with anything big in the next half hour.

I just...

I'm so tired of listening to people talking about how we've got to be using The Slot Machine That Sometimes Induces Psychosis, because other people are using it more, and we don't want to be using it less.

And meanwhile it makes statements that aren't even coincidentally correct.

And meanwhile meanwhile, I remain gobsmacked by the implication that the thing holding back coding projects is the amount of code that can be produced, or rather, the rate of such production, or rather, typing speed. Perhaps such sentiments simply originate from not paying attention to how code is actually produced; from my many project updates on this blog, I have simply started taking it as read that a novel design will take at least a week to flesh out, followed by a few hours of vigorous typing that will produce a rough artifact with a solid design, ready to refine along whatever metrics are desired.

But ultimately, it keeps circling back to, why the fuck is this being treated as an end in itself? The time when "a computer made this" was a solid advertising gimmick was apparently from 1985 to 1988 (all right, perhaps "a computer actually made this" had legs through the 90s), and if you want to claim some kind of productivity benefit, then you'd better figure out how to measure productivity, hm? Such knowledge will even be useful in a few years time when this whole thing messily implodes.

I don't want to be complaining like this, but I think the blame has to go on the people who will not let us stop thinking about this bullshit.

Anyway, I'm going to try to sleep soon.

Good night.

Tabletop Worldbuilding 2025-09-24

2025-09-24T04:00:00-04:00

This post isn't too related to the specifics of this project, but I realized something recently and I want to see if it makes sense as a way to approach projects like this. When it comes to developing a calendar, it's possible to end up considering multiple different kinds of day, month, and year, with intimidating-looking formulas to convert between them. Look at all of the nesting, and the subtraction hidden right in the middle. But if we switch our perspective from "the duration of a celestial event" to "the frequency of changes between instances of the celestial event", then the formula simplifies into a simple subtraction: the "solar day frequency" is equal to the "sidereal day frequency" minus the "orbital frequency". If we think entirely in terms of frequencies, then tidal locking results in a "solar day frequency" of 0, which feels a lot nicer to me than "the day length is... oops, divided by 0".

Looking at the various months associated with the moon, it's my impression that nearly every calculation involved in this can be simplified by considering the frequency instead of the period, and then not converting back until the end (when people would rather see "twenty-whatever days" than "four hundred-ish nanohertz"), or not even really then, simply dividing day frequency by month frequency, rather than month length by day length.

I'm a little tempted to take this in a crank direction of, like, "why don't they put it in these terms?" but for one thing, reciprocal-based formulas do seem to be known, and for another, like, nobody has a reason to listen to me on this beyond "hey, let's try it out", so, like, if you're doing calendar worldbuilding based on this kind of epicycle-inducing rotate-y stuff, give this a shot and see what you think. I will say that this seems much more suited for spreadsheets or some other programming language, than for manual computation, but then, manually computing this stuff is kind of a chore if you're not doing the calculations for their own sake.

Anyway, it's getting late and I should wrap up, just wanted to get some thoughts out.

Good night.

Weekly Roundup 2025-09-23

2025-09-23T04:00:00-04:00

Wednesday: I started really looking at the ruff check issues in my code, and disputed some of them.
Thursday: I kept on with that, but I also found and fixed a weird bug in my code.
Friday: I attempted malicious compliance with a lint error, which is not a good idea, and also brought in deal as a dependency so you can't tell I'm writing assert statements.
Saturday: I began to realize that I was using an old version of ruff, and way more lint errors appeared.
Sunday: The lints are done, so, um, now what?
Monday: Head feel bad.

Next week, I'm going to try to do more stuff away from my screen, whether that means putting down current projects, or figuring out how to get the pertinent details down on paper.

Diary 2025-09-22

2025-09-22T04:00:00-04:00

I thought I was doing all right today, but something is sapping my focus and making it hard to breathe, so, guess I didn't do anything worth writing about because my head hurt.

Um, guess I'll go brush now, and go lie down until the hurting stops.

Good night.

Coding 2025-09-21

2025-09-21T04:00:00-04:00

I got linting to pass, partially by making principled changes in response to the reports, but mostly by disabling lints that I disagreed with. (There was one that I disagreed with a lot.) Now, I can work on this Impliciula more, or pick up something else. Let's see what I can do here...

I can work on increasing coverage, but writing tests for existentials requires first figuring out whether I need to implement new value types.
I have a general idea for making test code shorter and a little less fiddly, but it needs a lot of planning and prep work to make sure it isn't a complete mess.
I can try to improve documentation, but that also requires planning, and also some of my generated docstrings might be a little screwy, according to Sphinx. And also there are just plainly some typos in this documentation because I tried to skimp on writing proper ReST.
For that matter, my naming conventions are kind of bad, and also not really conventions.
Trying to remember if there was anything else.

I definitely need a break now. Going to wrap up early and try to zone out with something unrelated.

Good night.

Coding 2025-09-20

2025-09-20T04:00:00-04:00

I'm almost done resolving lint errors, and once I figure out how to deal with this last batch of them, I just... need to resolve a few more. The "few more" are from pydoclint and—

Aw heck, I just realized I was focusing on src, and tests has like a hundred more errors.

...

I think it's time for another break. I'll figure this stuff out later.

Good night.

Coding 2025-09-19

2025-09-19T04:00:00-04:00

I'm still grinding through the lint errors. I ended up trying to go through all of the "missing docstring" errors on a per-function-name basis, because switching between them was too obnoxious, but that's left me in a state where I'm staring at a bunch of non-docstring errors and not quite sure what to do.

It doesn't want me to rebind self. So I changed it to self_ = ... and put a del self line after the assignment. This is better, I guess?

Next up is the various places I used assert statements. I'm trying to switch over to using deal, but looking over this, I'm a little confused as to why I only did this precondition check on a small number of method implementations. I'll have to think about this.

...

Okay, I got more and more lints taken care of, and now I'm running into stuff like "how do I document a protocol that's just like, 'you need to have this one method'?" I will try to figure this out later, because right now it is late and I need to sleep.

Good night.

Coding 2025-09-18

2025-09-18T04:00:00-04:00

I'm still poking at lint errors, and making slow progress. Part of the reason it wasn't faster was because I hit a bad interaction in all of the metaprogramming I'm doing.

Basically, for some node types, there's an inherited public method that forwards to an overridden private method, right? And I was trying to inject documentation from the private method into a copy of the public method. But I was doing this in __init_subclass__, which should have been fine, except that all of the leaf classes that need this are attrs classes, which gets you two classes per class. This is why all of my documentation was getting injected twice: first it gets injected in the non-slots class that is input to attrs.frozen, then it gets injected again when the method is referenced in the new slots class in the output.

I opted to fix this by simply keeping a list of private implementations on the public copy. This seems to work, but maybe I'll notice something weird later.

Anyway, it's late, gotta sleep.

Good night.

Coding 2025-09-17

2025-09-17T04:00:00-04:00

So.

I got started late, and I've got melatonin about to kick in, because for a while last night I just... didn't sleep, and I didn't like that at all.

Right now, I'm poking at all of the errors that ruff check is reporting, and disagreeing with a bunch of them, from somewhat valid reasons like "I'm not going to explain what __str__ is for here and now" to interesting reasons like "If I don't format the docstring in this bizarre way, it breaks the metaprogramming I just spent a few days on".

In any case, it's late, and I just got to "I feel like I have a very good reason to rebind self, but at the same time, I kind of feel like maybe some of these noqa comments need some explanation in the code?"

The most important thing is that it's late, and the melatonin has well and truly kicked in at this point.

Good night.

Weekly Roundup 2025-09-16

2025-09-16T04:00:00-04:00

Wednesday: I wasn't really focused.
Thursday: Same deal.
Friday: I still wasn't really focused, but I sort of talked about some things.
Saturday: I took some notes and made some plans for documentation-related metaprogramming in Impliciula.
Sunday: I took some notes on very niche worldbuilding stuff.
Monday: I put the metaprogramming into practice, and started addressing lint errors in Impliciula.

Next week, I'm going to keep on grinding out stuff for Impliciula, and probably do some writing that I may or may not post about.

Coding 2025-09-15

2025-09-15T04:00:00-04:00

I'm getting an early start on the entry today, because writing the entry means I have to work on a project to have something to write about. Anyway, here we go, documentation.

Python docstrings are stored exactly as they are represented in source code, and, if I recall correctly, reformatted as part of displaying help. This means that all of the __doc__ attributes I'm working with will have a trailing line of all whitespace, with no newline at the end. And a leading line with no indentation, and, because I'm generating method documentation, the remaining lines start at eight spaces worth of indentation.

Now, consider the parts of the docstring:

Short description
Long description
Parameter and return documentation

So, the parameter and return documentation should be on the "template" docstring, which should also contain the short description. Thinking more about the short description, I thought I wanted the name to be templated in, but I changed my mind. (Not putting the class names back.) This means I don't need to worry about the kinding helpers, and can focus on the typing code.

I want the private implementation functions to just provide the long description. If the implementations start with a blank line, have another blank line, and then the long description, then the "template" can simply interpolate the implementation documentation at the end of the first line, before the newline. Checking the behavior of strings when I copy and paste around, the final blank line of the implementation should be removed. (I suppose unless the template doesn't contain a newline, but it... should.)

All right, I've written a helper function to interleave all of this stuff properly. Now, I need to work out the "right" way to deep copy a function so I can give it a new docstring.

...

I found something on StackOverflow; it's probably fine.

The tests still pass, at least, although I did need to shove a bunch of typing ignores in.

Anyway, I figured I'd try to grind through the lint errors for a bit, and I need to ponder some of them before I move on, so I'm going to wind down for now and see what I think when I get back to it.

Good night.

Tabletop Worldbuilding 2025-09-14

2025-09-14T04:00:00-04:00

Let's just dump some notes on stuff in a category.

I've got a few solo campaign ideas in different systems kicking around, and one place where I tend to go above and beyond the recommendations of the rules—

Way too far, you mean.

—is in designing calendars. Dates, calendars, and holidays have always fascinated me and drawn my focus. This is an artificial system to keep track of natural events, but it also reflects and influences our culture, and even the world around us. I remember hearing about how some aspects of the weather are essentially on a seven-day cycle because they're influenced by human activity.

So, although it "is easy to keep track of" and "legible to general audiences" to either avoid dates, or to provide a relex of "normal" (read "Gregorian or maybe Julian") calendars when it comes to physical worlds, I like to see something more involved, in part because it situates the cultures that devised the calendar in the reality of their world, but honestly it's probably mostly that I like messing with the numbers.

With that background established, let's take some notes on the setting for a solo campaign that is... really not getting off the ground, for mysterious reasons...

(One brief disclaimer: none of this is admissable as a source for your homework or any other work; do your own research.)

I wanted to set the campaign on an Earthlike planet, but instead of giving it one moon-size moon, it has three smaller moons in Laplace resonance. The moons are all collinear once every sidereal month of the middle moon, so we ensure that it has an orbital period somewhat close to a month, to try to have something like spring tides. (This doesn't work astoundingly well because spring tides just rely on the synodic month when there's one moon, but having moons in resonance like this means that the sidereal months matter; possibly to deal with this, I will at some point endeavor to construct a synodic/sidereal lunar calendar specifically for tracking tidal effects.)

Because there are so many moons, and therefore also so many things that a moon could relate to, there are many possible options for calendars, moreso than the many options for calendars on Earth. The two initial ideas I came up with were a synodic calendar based around the inner moon with eight-day weeks, and a solar calendar that relates in some way to the synodic cycle of the middle moon, with six-day weeks. I haven't touched the latter in a while, and I've forgotten some of the details, so let's look at the former.

The days on this planet are slightly longer than on Earth, but near enough that using a twenty-four hour day results in an hour basically comparable to our own. This is what I call a concession to legibility. (And human-like physiology.) The long day just slightly overcorrects for a longer year, meaning that the tropical year of the planet is just over 361 and a third solar days. The synodic months of the moons are not-quite-sixteen days, not-quite-thirty-three days, and not-quite-seventy-and-one-half days.

So, each synodic month of the inner moon is just over two eight-day weeks. The accumulated error is very close to one day after eight months. To account for "sixteen days" sounding different from "a month", I call these sixteen-day periods "hextnights". Because "sextnights" sounds like a drastically different thing. I decide to have units of 8 hextnights called a "lunar season", in which one hextnight (probably the last one) is shortened by a day to account for drift.

Three lunar seasons is one or two hextnights longer than a tropical year, but I'd rather work with constant-length seasons, so I just accept that the lunar year is longer than it "could be". The lunar year is a decent approximation to the lunar cycle, requiring a further-shortened hextnight only around once every three hundred years. (This allows me to create a completely unintuitive meaning for the idiom "once a fortnight", and I'm very tempted to take that opportunity.)

I like this lunar year as a basis for a liturgical calendar, but I also want to adapt it do a solar civil setup. For this, I figured each solar year would be assigned 22 or 23 hextnights, depending on when the spring equinox falls relative to the full moon, and assigned a pair of numbers: 1 to 23 for the position in the year, and 1 to 8 for the position in the lunar season.

Putting this all together, we could end up with a date like: "CY 3307, Hx 13, 4/8, Waxing Week, Fifth Day (LY 4189, Fire Season)" (Or, if we go hardcore on the octal, "CY 6353, Hx 15, 4/10, Waxing Week, Fifth Day (LY 10135, Fire Season)") And I think that's neat.

Anyway, I spent way too long on this.

Good night.

Coding 2025-09-13

2025-09-13T04:00:00-04:00

Okay, let's see what I've got in the node classes. There are three classes that have two methods of interest each.

ValueType
CompTerm
ValueTerm

Each of them has a pair of inference/check methods, and the *Term classes have the template pattern kind of deal where there's a private abstract method, with a public interface that calls the private method with some boilerplate. In every case, there are predefined method implementations.

In every case, I want to have some way to substitute the class name into the documentation for the predefined method. And in the *Term types, I want the public documentation to reflect what's written on the private implementation.

Before I think too hard about this, I'm going to make the change that this all suggests: properly naming all of the *Type subclasses, rather than just relying on the fact that they're unique within their module and just calling them "Type".

The next step, which I won't take tonight, is to write __init_subclass__ methods on the classes, and create copies of the public methods of interest, with updated documentation. This is another case of "something I need to think carefully about", so maybe I'll switch gears for a bit tomorrow. We'll see. For now, bed.

Good night.

Diary 2025-09-12

2025-09-12T04:00:00-04:00

I did some investigation into stuff I'll need for documenting Impliciula, but I didn't take any of the notes that I need to properly plan it.

I did, however, mess around more with worldbuilding stuff. I'm doing stuff with fictional calendars, and I've got a bunch of fictional but (probably) basically mathematically sound orbital parameters, and I'm using them to work out good patterns of intercalation. I just now wrote some helper functions for dealing with continued fractions. Now I need to work on getting a better idea of how good an approximation needs to be; I've got some approximations that are good in the short term, but start getting notably off after around 300 years, at which point the fact that they used to be good is no consolation.

If I have proper focus tomorrow, I can put in more work on one or both of these, or on one of the other projects I have drifting around. For now, I really need to get to bed.

Good night.

Diary 2025-09-11

2025-09-11T04:00:00-04:00

I was pretty sore and unfocused today; I ended up just tracking down some old worldbuilding stuff that I want to mess with. There's nothing much to say for now.

I'm dragging my feet on getting ready for bed, but I should just do it.

Good night.

Diary 2025-09-10

2025-09-10T04:00:00-04:00

I was thinking about stuff to do for coding, but I ended up zoning out and taking things easy. Maybe I'll work on things tomorrow, maybe I'll keep taking things easy. Either way, I should wrap up now before I have problems.

Good night.

Weekly Roundup 2025-09-09

2025-09-09T04:00:00-04:00

Wednesday: I didn't really do things, so I tried to plan things.
Thursday: I picked working on MOTR, and ended up just updating it to maintain Mypy compatibility. I did figure out what I want to cover, but not how to cover it.
Friday: I didn't do much, but I did write some stuff up that I didn't show off.
Saturday: I did the easy test updates for simply-typed lambda calculus and system f in Impliciula.
Sunday: I updated the remaining tests, and tried to work out how existential types do useful things.
Monday: I hopefully fixed a bug around existential types, and tried to figure out what my next priority should be for Impliciula.

Next week, I will either endeavor to properly document Impliciula, figure out some aspect of what is going on with MOTR, or do something else entirely.

Coding 2025-09-08

2025-09-08T04:00:00-04:00

I confirmed that I should have known about the issue I found with existential types, and put together a fix (untested) for it. Now, I'm considering where to go next.

Well, it would be nice to get more coverage on the existential types module than just importing it.
But, important question, can I get any meaningful tests written when I just have the unit type?
If not, before writing more types, I want to actually try to dig myself out of the documentation hole I'm in.

I don't want to mess around with this right now, so I'm going to wind down now.

Good night.

Coding 2025-09-07

2025-09-07T04:00:00-04:00

The tests all pass, I've got 93% branch coverage and 85% mutation coverage, and, while there are various things that make sense to do, right now I'm focused on, what even is the existential types code doing? So, let's crack it open, and see if I can sort it out into something that makes sense to me.

The existential types module defines three classes:

Type, a value type representing existential types.
Pack, a value that can be checked against Type.
Unpack, a computation that wraps a value.

Let's start with Pack. An instance of Pack is just a pair of a value type and a value. The value type and value are not inherently related.

Instead, given a Type, which combines a type variable, the kind of that type variable, and a value type expression, the Pack is an instance of the Type if, when the Pack's wrapped value type is substituted for the Type's type variable in the Type's value type expression, the Pack's wrapped value can be checked against the resulting value type. Clear as mud?

Now, to actually use a Pack in computation, we need an Unpack, which has an absurd number of attributes. The key thing is that it allows constructing a Type (this differs from a pure type inference setup, in which the Pack must specify its own type), has a variable associated with it, and contains a Pack, as well as the actual computation, which has the variable and type variable free. Evaluating the Unpack involves substituting the Pack values for the variables in the computation, and then evaluating that.

To make this of general interest, note that the contained Pack can actually be a variable, in which case the Unpack could be, for example, the body of a function, and then different behavior could be obtained by passing in different Pack values, which are only constrained to match the Type of the function argument; the packed values and types can vary freely as long as they conform to the Type. This is backed up by the fact that the inferred type of an Unpack node is independent of the contents of the packed value.

However, for the type inference to work, the Unpack's variable has to be bound to the value type expression without substituting any type for the type variable, and then the computation must infer a type that doesn't depend on the type variable, which... hm. My code might be wrong? But subtly.

Figuring out what makes sense requires some kind of actual example.

All right, let's suppose we have a p = Pack(unit, Thunk(Abstraction(Variable("x"), Return(unit(None))))). Among other Types, this satisfies Type(T, TYPE_KIND, ThunkType(FuncType(T, ReturnType(T)))). So we can have Unpack(T, TYPE_KIND, Variable("f"), ThunkType(FuncType(T, ReturnType(T))), p, UMMMM).

Okay, that didn't work, let's try again. p = Pack(unit, Thunk(Abstraction(Variable("x"), Return(bool(True))))). Type(T, TYPE_KIND, ThunkType(FuncType(T, ReturnType(bool)))). Unpack(T, TYPE_KIND, Variable("f"), ThunkType(FuncType(T, ReturnType(bool))), p, OKAY LOOK WE'RE GETTING CLOSER).

Part of the stumbling block here is that I've been leaving record types as an exercise for my future self, so it's sort of hard to shove enough information into a Pack with the tools I have currently. So, let's try living in the future.

p = Pack(unit, {x: unit(None), f: Thunk(Abstraction(Variable("x"), Return(bool(True))))}). Type(T, TYPE_KIND, {x: T, f: ThunkType(FuncType(T, ReturnType(bool)))}. Unpack(T, TYPE_KIND, Variable("r"), {x: T, f: ThunkType(FuncType(T, ReturnType(bool)))}, p, Application(Force(Attr(Variable("r"), "f")), Attr(Variable("r"), "x"))).

So, the question is, ignoring that some of these nodes don't exist, can the Unpack be successfully typed as ReturnType(bool)?

...

Okay, it looks like the typing should work, but I've just realized that my actual concern is around the determination of free type variables, and whether that's an issue is a very definite maybe.

Okay, here's what I think needs to happen: type inference of Unpack must fail if the type of the computation has the type variable free. This means I need a new constraint type. Yaaaay. I'll note this down in the code and then go to sleep.

Good night.

Coding 2025-09-06

2025-09-06T04:00:00-04:00

I ground through the tests in Impliciula, getting them to pass, although probably also making them not as good at catching mutations. I'll work on that once I've managed to get the coverage up. And I'll consider that once I've updated the remainder of the tests.

See, a bunch of the System F tests have this pattern of "check one expression against the inferred type of another", and I need to take some time to decide whether to bash together a helper function to deal with the whole "they're generators now" thing, or to "simply" work out the actual type they should have, and check against both of them separately.

I'm not going to make a decision right now, because I'm still really tired and stuff and I want to get to bed.

Good night.

Diary 2025-09-05

2025-09-05T04:00:00-04:00

Today was rough on me. I thought some about worldbuilding, and also worked on character backstory for a Dungeons and Dragons campaign.

I didn't do anything else that translates into any kind of writeup, let alone one as detailed as I'd like.

I'm going to try and get some rest and actually take things easy.

Good night.

Coding 2025-09-04

2025-09-04T04:00:00-04:00

Okay, first off, it's late, and I'm going to try to make this quick. And probably fail.

Second off, I decided to take another look at MOTR. In the intervening months since last time, Mypy made some bug fixes and improvements, and I had to update the code and tests to account for it. Once that was out of the way, I took another look at the coverage stats, and I... do not remember what I was doing. Something about needing to rewrite the wrapper code so that the module globals are changed to properties on a class, but first I need to get more test coverage... somewhere? In particular, I need to make sure that each instance gets unique copies of the labels... Looking at the uncovered lines, the most plausible thing is... the new matrix code. Oof. That's going to take some careful study and review. Definitely not happening right now.

Got to wind down.

Good night.

Diary 2025-09-03

2025-09-03T04:00:00-04:00

I'm getting stuff done on different projects here and there, but once again, none of it is really adding up to something that feels worth a post.

Let's see, do I want to commit to anything for tomorrow?

If I'm up for a straightforward grind, I can try to fix the current test suite for Impliciula.
If I'm up for open-ended design work, mess with the synth code.
If I want to wonder what the heck I was doing, pick MOTR back up I guess.

For now, I need to wrap up for the night.

Good night.

Weekly Roundup 2025-09-02

2025-09-02T04:00:00-04:00

Wednesday: I mentioned a worldbuilding project I've been messing with.
Thursday: I didn't have anything to say.
Friday: I made a category for the worldbuilding project, and tried to describe it in a little more detail.
Saturday: I played a bunch of Pipistrello and the Cursed Yoyo.
Sunday: I started contemplating specific bits of technology that might exist in the setting.
Monday: I thought some more about the potential evolutionary history of these weird guys I came up with.

Next week, I'm really not sure I'm over the migraine stuff from the past week, so I'll be trying and probably failing to take things easy.

Blind Parasites Nesting in the Insensate Flesh of an Unthinking God 2025-09-01

2025-09-01T04:00:00-04:00

Wow, I really played myself by putting the full title in the filename, huh.

Anyway, I'm slowing narrowing down what I expect out of the spec-evo side of this project. Triploblasty seems reasonable, closely replicating the evolution of the spinal cord does not. The selection pressures associated with cephalization currently seem to me to be weaker, which is not the same as absent. Unsure what kind of jaw structure is plausible. I've got some weird ideas about manipulatory appendages that I don't know if they make sense. The idea being to have hand-like structures, and those have bones, but the "arms" have inflatable hydraulic chambers instead of bones, allowing them to switch between "flexible arm movement for propulsion" and "fine motor control and high leverage for manipulating objects". Probably need bones at the joints between chambers. As I said, not sure if this all makes sense. I don't know if it would plausibly evolve even if it did make sense in terms of fitness.

Anyway, I don't want to think about it any more tonight.

Good night.

Blind Parasites Nesting in the Insensate Flesh of an Unthinking God 2025-08-31

2025-08-31T04:00:00-04:00

I tried to take things easy today, with some... mixed success. But I came up with some new thoughts about this setting, so I'm going to relate them in order to have something to talk about.

I plan to make my first work in this setting a "translation" of an in-setting document, a bestiary, and that means that I need to ponder questions like:

What forms does communication take in a setting without air or light?
For that matter, how does perception work?
What kinds of vocalization could exist, and what would make sense for communication?
How would writing develop under such constraints?

Thinking about this, I ended up with the mental image of a writing system based on knots, like quipu, but going beyond "numbers and accounting, probably" to somehow convey thoughts more generally, and also the idea of physically connecting different strands; in other words, having the original format of this medieval-inspired bestiary be hypertext. I've got the mental image of encyclopedias so big, that they have an architectural structure supporting and organizing them, and allowing authors, editors, and readers to swim around inside the "book".

Nailing down a lot of these concepts, however, will require coming up with at-least-plausible-sounding answers to questions like "how do these different forms of... meat-protist actually develop their gross anatomy?"

Anyway, it's late and I want to stop staring at this screen.

Good night.

Diary 2025-08-30

2025-08-30T04:00:00-04:00

So much Pipistrello today.

I don't think I have anything else that I want to discuss right now, and it's late, so I'm going to wrap things up for now.

Good night.

Blind Parasites Nesting in the Insensate Flesh of an Unthinking God 2025-08-29

2025-08-29T04:00:00-04:00

Oh wow, my text editor is not a fan of how long this title is.

Anyway, this is a setting that I apparently came up with about a year ago, and shelved for a while, and I've been revisiting it recently. The basic premise is that, in this setting, instead of stars surrounded by planets, there's just an endless expanse of flesh that lies dormant except in the vicinity of sun-sized hearts, which pump blood that gets filled with "life". This "life" is fed upon by a strange ecosystem that developed within one such circulatory system. The basic idea is to do spec-evo/bio worldbuilding in a fantastical setting where everything is made at least a little out of meat. I've got a very small idea of the kind of life that could exist in this context, but I'm having trouble figuring out how I want to convey it. One idea that I'm looking into is, rather than a documentary/"scientific" perspective, present things as an in-universe document similar to a medieval bestiary. However, I don't want to reinforce that it's a bestiary through the usage of affected antiquated language or spellings; I don't want to write stuff like "beeynge" or whatever. I guess there are sites that rephrase and summarize bestiaries in contemporary language, so that seems reasonable as a thing to try.

Anyway, I'm trying to take things easy right at the moment, so maybe I'll be up for trying that out in a few days. For now, I need to wind down.

Good night.

Diary 2025-08-28

2025-08-28T04:00:00-04:00

I've been thinking more about different projects, but I haven't got enough done today to write anything up, and I don't have the focus to do so if I did. I just so very much want and need to rest my eyes.

Nothing else on my mind right now.

Good night.

Diary 2025-08-27

2025-08-27T04:00:00-04:00

I need to switch gears away from Impliciula for a bit, for my own good, but I'm not doing great at it.

I tried to get to a worldbuilding project from a year ago, but we'll see how I do with that. I'm right now just focusing on consolidating and polishing my notes, but there honestly aren't much in the way of notes, so I'm going to have to work on writing or drawing new stuff soon. I'm going to need to figure out how to share that, or do something with it.

For now, though, I've got to wind down and get to bed.

Good night.

Weekly Roundup 2025-08-26

2025-08-26T04:00:00-04:00

Wednesday: I switched my focus back to Impliciula, and made some progress getting the new code to typecheck.
Thursday: I had an idea for handling coercion.
Friday: I had a better idea for handling coercion.
Saturday: I made some more progress.
Sunday: I got most of the way done with the typing side of things.
Monday: I finished up the typing side of things (that is, making sure Mypy is happy), and decided I needed a break before trying to get the tests to pass.

Next week, I don't know, trying to actually sort of take things a little easy, I guess?

Coding 2025-08-25

2025-08-25T04:00:00-04:00

Okay, everything type-checks once again. And by "everything", I mean "everything in the src directory". If I check the tests...

Found 150 errors in 2 files (checked 6 source files)

Is that bad?

Shut it.

Anyway, while I could just barge in and start spot-fixing things until it passes and sort of makes sense, now would be a really excellent time to ponder the ergonomics of these syntax trees. Things to ponder include:

I realized that requiring bidirectional typing to always be valid doesn't work under reduction (and, fair point, the paper stated this and I just ignored it until the pain was obvious), but... suppose I had a parallel set of types just for constructing the tree, which could then generate the tree meant for typing and reduction, and have it guaranteed that the tree would at least generate a specific error rather than just... giving up.
Working in possibly the opposite direction, the syntax for constructing a tree right now is painfully verbose. And I don't have a clear idea of what to do about that.
If I do have two tree structures, perhaps I can generate one from the other somehow.
Alternatively, I'm a little tempted to see if building the tree from a concatenative language would make things nicer, but I doubt it.

Anyway, I took way longer with this writeup than I planned, so I've just got to cut it off now.

Good night.

Coding 2025-08-24

2025-08-24T04:00:00-04:00

Tired...

Regardless, I have gotten most of the way through rewriting these modules, assuming I didn't mess things up with the more verbose style. All that's left is to update things for universal types.

I haven't done that yet because it looks like I need some new bundle methods to make it really work, and I only had the focus today to grind through the rest of the type errors, not to write a bunch of new code, even if it should be pretty trivial.

If I have the focus for it tomorrow, my plan is to write up all of the methods I need to update, and figure out how to support them. Right now, I need to wind down.

Good night.

Coding 2025-08-23

2025-08-23T04:00:00-04:00

I tried to take things easy today, but I did get some code updated to do constraints-based typing. And fixed the problem with the code I pasted yesterday. (It should have emitted constraints, and didn't.)

Other than "there were bugs, I fixed them", I don't have anything terribly interesting to say about this code, besides that I ended up using with (yield from, which sounds kind of fake to me right now. Also, because I'm, like, making it more explicit what's happening with this code, it is getting really verbose.

I've got nothing else to say, so I'm going to wind down.

Good night.

Coding 2025-08-22

2025-08-22T04:00:00-04:00

I know there are some typos in the code from last post, but the bigger issue is that typing descriptors is... frustrating, and I ended up going with a solution that was just simple enough that Mypy couldn't totally biff it:

class Failing(Protocol):
    @classmethod
    def failure(cls) -> Self: ...


def coerce[T: Failing](
    cls: type[T], bundle: Bundle[Any]
) -> Constraints[Bundle[T]]:
    obj = bundle.data
    if isinstance(obj, cls):
        return Bundle(obj, bundle.metadata)
    yield from ()
    return bundle.extend(lambda _: cls.failure(), CoercionFailure())

With this, I have what I need to be confident updating the rest of the nodes. I think.

Anyway, it's late and I'm tired.

Good night.

Coding 2025-08-21

2025-08-21T04:00:00-04:00

All right, let's see about that descriptor concept. It's going to look something like:

@attrs.frozen
class Coerce[T]:
    ctor: Callable[[type[T]], T]

    def __get__(
        self, instance: T | None, owner: type[T]
    ) -> Callable[, Constraints[Bundle[T]]]:
        def coerce(type_bundle: Bundle[Any]) -> Constraints[Bundle[T]]:
            typ = type_bundle.data
            if isinstance(typ, owner):
                return Bundle(typ, type_bundle.metadata)
            yield from ()
            return type_bundle.extend(
                lambda n: self.ctor(owner),
                CoercionFailure(),
            )
        return coerce

Okay, let's ponder this. I'm feeling okay about this, but I'm assuming that I'd need to tweak this some to make it actually work with Mypy.

Anyway, I'm going to sleep on this, and get to bed soon.

Good night.

Coding 2025-08-20

2025-08-20T04:00:00-04:00

I plinked away at the System F stuff in Impliciula for a little bit. I've now got the existential module passing type checking, and I could move on to the function module, but I really want to get the generic coercion idea working, and I need to take some time to decide whether that's worth pursuing right now, or if I should just copy and paste the code, and change the one line that needs to change.

For right now, I should get to bed, but it's nice to confirm that it actually is possible to get a file to pass type checking with all of the... stuff... that I'm doing to the code. Oh, wait, I never did decide whether to include a coercion failure node. Let's think about that some more while I get ready for bed. Anyway, better move in that direction. (Actually, I think I will do the coercion failure node real quick, while this publishes.)

Good night.

Weekly Roundup 2025-08-19

2025-08-19T04:00:00-04:00

Wednesday: I put down some thoughts about synthesis, mostly additive synthesis.
Thursday: I made a little progress on the lower-level stuff for Impliciula, and then ended up not too enthused about continuing just yet.
Friday: At the last moment, I figured out how to get the synthesized sound to be more pleasant to me.
Saturday: I made some progress, but the main thing I got was that my prototype synthesis code has some serious limitations.
Sunday: Not much happened.
Monday: I took a bunch of notes on how to improve things with the synthesizer.

Next week, I will either figure out how to actually fix up the synthesizer code, or be up for continuing on Impliciula, or some other thing. I dunno.

Diary 2025-08-18

2025-08-18T04:00:00-04:00

Little bit of a weird day today. I had the time to sketch out high-level designs for the synthesizer stuff, but I don't feel like refining my notes into anything actionable yet.

Basically, the code that currently exists doesn't really have a concept of "instrument" in a useful way, and can't meaningfully ornament notes. I have a general plan for addressing these shortcomings, and trying to build up towards supporting higher-level ideas, like phrases.

I'm going to wind down for now, and I don't have a good segue for that.

Good night.

Diary 2025-08-17

2025-08-17T04:00:00-04:00

I kind of poked at stuff today, but mostly I ended up zoning out until just now, and realizing that I am up way late and I haven't written anything. I won't accomplish anything if I try now, so I'll just post this.

Good night.

Coding 2025-08-16

2025-08-16T04:00:00-04:00

Okay, I didn't have the focus to look at Impliciula-related stuff today, but I did mess around with coding related to audio processing. I've managed to put together some stuff just by typing a lot, but there are some very obvious limitations in the interfaces I have currently, so I'm trying to figure out how to address them.

The actual musical content is some xenharmonic nonsense that I might as well try to describe. Basically, I've been reading about dissonance curves and thinking about the implications of them. I realized that if the disonance minimum around the third is shallow instead of steep, then there won't be a strong draw to a precise value, which means that a spectrum that lacks fifth harmonics won't have a specific, preferred third, and would allow for neutral thirds; building a scale from neutral chords ends up suggesting 7TET instead of 12TET. Now, I didn't really like the sound of "basically a sawtooth wave but with certain harmonics suppressed", so I ended up switching to using undertones instead of overtones in my additive synthesis. This results in a sound that... is a sound. I like it, maybe I'll try messing with it a little later, but that's not the priority currently. I slapped together an ADSR implementation, and, like most of this project, got Numpy to do the heavy lifting; I don't know if the parameters I'm currently using are the greatest, but I can mess with those some other time, as well.

I initially assumed that the use of undertones in the synthesis meant I would need to put the melody in the bass and invert chords from 12TET (ignoring that it's not possible to make a 7TET chord sound "different", per se), but putting the melody all in the bass was too subtle for my taste (or maybe just for my laptop speakers; it was fine-ish over headphones), so I switched everything back.

At this point, I technically have what I need to try out more ambitious compositions than "the 7TET chromatic scale" and "pop music loop but it sounds weird because all of the thirds are different and also the fifth is even flatter than 12TET, like by a decent amount", but. Important concepts like "an instrument" are not meaningfully surfaced, and I'm going to need to do something about that in order to have a chance of moving from "arduous copy and paste" to "handy functions" or even "algorithmic composition".

Anyway, my focus petered out there, so we're done for tonight.

Good night.

Diary 2025-08-15

2025-08-15T04:00:00-04:00

I ended up messing with additive synthesis today, with an eye towards doing something xenharmonic with an instrument with a harmonic spectrum. I got all of the technical stuff I wanted put together, and now I'm floundering with the fact that everything I've done with the spectrum results in a sound I would describe as "like a pipe organ, but a little more annoying". I've got a lot to learn about controlling timbre, clearly.

(Maybe I should just make it really inharmonic?) (Huh, just going full negative harmonic gave me something nicer. Good to know.)

Anyway, I'd better wrap up.

Good night.

Coding 2025-08-14

2025-08-14T04:00:00-04:00

I've been messing with music stuff (and Pipistrello) most of the day, but I did just now hammer out a rough prototype for the placeholder, or "failed" node classes. I defined the classes, and resolved some TODOs, and this has gotten me basically right up to the point last time where I said "and then I'm going to have to think about it".

There are basically a couple of things going on here.

Bonus thing: the more inference methods I write, the more boilerplate I notice, and wonder if they should be written slightly differently.
The new "failed" nodes could in theory be inserted "manually" into the AST, which would break the preconditions for type checking and inference, but do I want to put in the effort to surface that this has happened? What is a good/right way to handle this, in terms of generating constraint failures?
I've got most of the infrastructure I need to create synthetic nodes when coercion fails, but do I want to insert a path segment to indicate the coercion? To be clear, the success case doesn't have a new segment, and I think that is correct.
Can I formalize any design constraints on how type constraints are generated? Are there assumptions I can rely on about where different kinds of node came from?

Having written all of that down, I think the sensible way forward is to take a conservative approach, in various senses but not the political one, ew, cringe, and then, once I have a working prototype, start documenting and refining it.

For now, I've got to wind down.

Good night.

Diary 2025-08-13

2025-08-13T04:00:00-04:00

I kind of expected that I was going to get more work done today, but it seems that my focus and motivation have been a bit low.

As I mentioned, I have been looking into music and synthesis some, and I've got some notes on how to put stuff together:

Instruments aside from, like, pure percussion have a repeating steady-state.
This means that they can be approximated with harmonic additive synthesis.
But getting all of the higher harmonics right is a pain.
There are two ways to nudge a waveform without disturbing the period:
Phase modulation by a wave with the same period;
Amplitude modulation by a wave with the same period.
These can be done independently, and also waves with the same period can be linearly combined.
I knew of all of this in the context of the major synths, except I hadn't considered the stuff, which I'm still not totally sure I get, about coupling resolving inharmonicities. Like, it would be nice if that were a thing, but if it were, why do pianos need to be tuned like that?

The thing to investigate is what subtle phase modulation does to a pure tone, and to see how things change with more harmonics.

For now, once again, I've got to get ready to sleep.

Good night.

Weekly Roundup 2025-08-12

2025-08-12T04:00:00-04:00

Wednesday: I put down some thoughts about Villeins of Tomorrow.
Thursday: I collected published RPGs to look at. (I'm going to need to pick up some non-fiction stuff as well, but I haven't done that yet.)
Friday: Migraine.
Saturday: Migraine.
Sunday: Poked a little at Impliciula.
Monday: Researched music-related-stuff.

Next week, I'll try to avoid backsliding on health, and secondarily make progress on other stuff.

Diary 2025-08-11

2025-08-11T04:00:00-04:00

I ended up taking things easy again today, so there's not much to talk about when it comes to code. Or RPG concepts.

I have been looking more into music-related stuff, and letting ideas percolate there. Unless I get really inspired, I'm going to have to accept some actually very reasonable compromises, but I'm not ready yet.

For now, I should wind down.

Good night.

Coding 2025-08-10

2025-08-10T04:00:00-04:00

I didn't have the focus for working on Villeins of Tomorrow today, so I ended up doing some quick work on Impliciula's System F layer. My goal was just to implement type substitution for bundles, which allowed me to update more of the existential nodes to use bundles in type checking.

I am too tired to properly explain what all of that meant.

Anyway, the next step in updating is to finish implementing type coercion, that is, coercion between types that are different types. That requires a few things:

Placeholder types to handle constructing a coerced type.
Rework the equality constraint to consider placeholder types equal to everything, because, like, if you're comparing against a placeholder type, then that was generated from a constraint failure, so, like, you don't need more constraint failures. I can hammer out the details later.
See if I can make a dedicated descriptor for coercion, because it kind of seems like it should be possible to hardcode most of the behavior. Just need to... construct a default value... from the class... that the descriptor is being defined on. I'm pretty sure I can make this work somehow.

Well, that's a fun task that I just set for myself. We'll see about that some other time.

For now, sleep.

Good night.

Diary 2025-08-09

2025-08-09T04:00:00-04:00

I thought I was feeling better this morning, but either I was wrong, or work made me wrong.

Didn't get anything done, going to go lie down now.

Good night.

Diary 2025-08-08

2025-08-08T04:00:00-04:00

My head hurts and I really don't want to prolong things tonight, so I'm going to wrap things up here and load up my Villeins of Tomorrow research on my e-reader so I can look at it without getting eyestrain.

Nothing else to think about right now, so let's go.

Good night.

Villeins of Tomorrow 2025-08-07

2025-08-07T04:00:00-04:00

I was pretty tired and out-of-it today, so I ended up just trying to find sources of inspiration. Everything I'd thought about offhand earlier, I was able to track down in my various libraries, so now I just need to ponder what else to look for, and look over what I have so far.

I think the thing to do is to take notes, but I don't have the focus for it at the moment, so I'm going to wind down now and see how I feel tomorrow.

Good night.

Villeins of Tomorrow 2025-08-06

2025-08-06T04:00:00-04:00

Actually, you know what, let's actually go into some detail here before I throw myself into bed.

With Villeins of Tomorrow, I'm trying to comment on or reframe a lot of the aspects of pulp science fiction that didn't age well. To be a little more concrete than that, I am in favor, for now, of space battles where describing them sounds like being trapped in a kaleidoscope. Not a fan of ancient eugenically-manipulated ancestral lines of ultimate space narcs.

So I've got this idea of different factions representing different strands of pulp science fiction (construed somewhat broadly), and player characters who clash with these factions as a result of their weird, problematic aspects. What I want to get a handle on right now is, what do the player characters do? Both in the sense of "what kinds of objectives do they pursue?" and "what tools are they given to pursue those objectives?"

Putting it out like this, I guess the solution is to come up with a bunch of distinct character concepts, situate them within the pulp milieu, and see what feels right. One thing that it occurs to me I'll be doing a lot is figuring out which "strengths" are actually weaknesses, and vice versa.

Places to look for mechanical inspiration: Apocalypse World and other PbtA games, Blades in the Dark and other FitD games, and, uh, Golden Sky Stories. Trust me on this.

Anyway, I can't pursue this any more tonight.

Good night.

Weekly Roundup 2025-08-05

2025-08-05T04:00:00-04:00

Wednesday: Didn't get much done.
Thursday: Still didn't get much done.
Friday: I made some progress on my efforts to rework typing in the System F layer.
Saturday: I managed to deal with the problems I was running into, so what's left now is to grind through all of the type errors.
Sunday: Okay, so, grinding through the type errors is going to be more tedious than I expected.
Monday: So, let's see about taking a break for a bit.

Next week, I'm going to try and focus on Villeins of Tomorrow for a bit. I have some ideas for Impliciula, but they're not all-the-way baked yet.

Diary 2025-08-04

2025-08-04T04:00:00-04:00

As I said, I'm trying to focus on stuff besides Impliciula, because now that I've gotten past the blockage, I'm sort of feeling the strain from pushing. As such, I've started working on yet another project.

:) (glitchy)

Of course you have.

Anyway, there will probably come a point where I decide to take a break from this new project as well, but that's maybe sort of the idea.

This new project is called Villeins of Tomorrow, and there are a few directions I could take it, but the plan right now is to make a PbtA game inspired by classic pulp science fiction; part of the idea here is to see what I even think of it once it comes together thematically.

Thematically? You're going to make some people really angry, aren't you?

Don't worry, they deserve it.

Anyway, I'm tired and I want to get to bed.

Good night.

Coding 2025-08-03

2025-08-03T04:00:00-04:00

I was mostly focusing on stuff besides Impliciula today, which was... probably good. However, I did just now take a look at the state of type errors with the rewrite, and I have found things that I was ignoring. Basically, I didn't think too hard about how to translate various operations from purely node-based, to base on nodes and bundled metadata. I'd sort of glossed over this with how part of inferring the type of an unpack operation involves synthesizing an existential type from various fields on the unpack node, because all of those fields were from the same node, so whatever. (That previous sentence may have sounded weird. I will not be explaining further.) But now I'm working on checking the type of a pack node, and suddenly I need to perform type substitution with values from two different bundles, and I'm unclear on what the desired semantics even is. Like, do I define some kind of merge operation on paths, or is there some sense in which one of them "wins"?

Thinking about how it'll be used, it's kind of as if the path needs to introduce a variable binding to the bundle that's substituted in? After all, it would be good to know where the value came from, except that currently the substitution is so thorough that it's not obvious, except, except. If the path description is good enough to follow through the nodes (this is a big assumption at present), then it will reach a variable, and if there is a binding to that variable, then there shouldn't be a problem, per se, with using that binding to continue.

So, we need the binding that reaches the variable, obviously, and we need the binding that reaches the value, do we need to track "why did we substitute that variable specifically"? My gut feeling is that it would be very unwise to make a snap decision here, so I am going to try to wind down now, and focus on other stuff for a bit. We'll see how that goes.

Good night.

Coding 2025-08-02

2025-08-02T04:00:00-04:00

All right. I figured out the answer to that protocol stuff: just give up and hard-code the possible types. This is frustratingly inelegant, but I can't deny how much easier it was overall.

I then ran into "my whole concept of reifying node transformations and bundling them with a path component, does not work, type system cannot handle it, totally breaks things", and I figured out an acceptable alternative. At that point, it actually became feasible to rewrite the implementations for the concrete node types. And the code is totally normal and not-at-all-cursed.

Uh-huh.

At this point, I'm making my way into "draw the rest of the owl" territory, which is actually an improvement, because where I was before was "all of your drawing implements got snapped in half". Now that I have means of handling all of the basic primitives, and it seems to be compatible with Mypy, there's less of a "mad scramble to fix things" vibe, and more of "so, out of various valid options, which do I prefer?"

As such, I will be trying to take things easy, now that I've figured the big stuff out, and part of that is winding down now, even though Bandcamp Friday didn't go quite right.

Good night.

Coding 2025-08-01

2025-08-01T04:00:00-04:00

All right. Here we go.

I've figured out where some of the Mypy problems I was hitting were coming from, but I'm having trouble figuring out what to do about them.

Basically, I'm creating a wrapper class that forwards some calls to its wrapped object, and this is made to typecheck by relying on explicit self annotations and protocols. The problem I've run into is that typing a function with def my_func[T: Proto](self: Bundle[T], ...) -> ... seems to work fine, but if the protocol is a little fancier, like Proto[T], then constraining the type variable doesn't seem to work, and the best I can do is def my_func[T](self: Proto[T], ...) -> ..., which I thought this class was covariant, thought it would Just Work, but evidently not.

I'm not going to figure this out tonight, so I'll just wait and see what I come up with.

Good night.

Diary 2025-07-31

2025-07-31T04:00:00-04:00

Okay, I need to give myself something else to work or chew on, because fixing up type error reporting for Impliciula is just not happening. And this should be the easy part!

Let's see, I'm roughing out yet another setting to potentially do something in... I'm trying to get a handle on phase modulation and potentially way overthinking things still...

Maybe I'll have some time and focus to think about this tomorrow, but for now I have to take things easy.

Good night.

Diary 2025-07-30

2025-07-30T04:00:00-04:00

I had trouble getting to sleep last night, and then I had trouble focusing on anything today. It would be a very bad idea to try to force anything, so I'm going to wrap things up and try to get to bed earlier.

Can't think of anything else to say.

Good night.

Weekly Roundup 2025-07-29

2025-07-29T04:00:00-04:00

Wednesday: I ran into trouble with my first attempt at threading in context.
Thursday: No energy.
Friday: I gave up on some of the fancier ideas I was trying to make happen.
Saturday: I ran into some issues implementing the "simplified" ideas.
Sunday: I got some work done, and paved the way for more work.
Monday: I tried to focus again on giving access to context when generating constraints, and ran into the problem that none of my ideas so far are really appealing.

Next week, I'm going to try to find something I'm at least willing to try, and maybe try splitting focus between projects for a bit.

Coding 2025-07-28

2025-07-28T04:00:00-04:00

Today, I forgot about how I wanted to get kind checking done right-er, and focused on nothing in particular, but I did just now try to update some of the code that I maybe want to use to thread contexts through the typing code. I'm thinking about that, and kind of vacillating between "Well, I need to try something." and "I really don't want to try anything that I've considered so far."

So, once I have a bit more time and energy, I'm going to try to come up with more ideas. For now, I've been too slow getting ready for bed. I'll get on that now.

...

Okay, I'll get on that now.

Good night.

Coding 2025-07-27

2025-07-27T04:00:00-04:00

If I play my cards right, I'll get to bed at a somewhat more reasonable time.

Today, I took things easy, and then focused on dealing with various papercuts I'd identified around the typing code. I finished bidirectionalizing the kinding, and this dealt with a bunch of small annoyances. Going from there, I could take another crack at figuring out constraint contexts, but there are some bits of code that I haven't fully worked out what they "should" do, which are now a little bit easier to rewrite once I know what I want from them.

So, my plan is to wrap up quickly, then review the sources I'm using while I wind down in bed. Let's see about that... right now.

Good night.

Coding 2025-07-26

2025-07-26T04:00:00-04:00

It's late, so quick rundown of the state of things:

I rolled back to what I believe is the last version that type-checked.
I reintroduced some of the typing improvements I made on what is now a weird offshoot.
I'm trying to figure out the right way to signpost "some of these subclasses are just for ease of subclassing, do not annotate anything with them".
I'm still letting the question of how to associate this context data percolate in my mind.

I can't make much progress without having an answer to one of those questions, so I think I also need to be open to switching projects for a bit, if I'm really unsure. Regardless, need to get to bed right now-ish.

Good night.

Coding 2025-07-25

2025-07-25T04:00:00-04:00

I did a bunch of planning work elsewhere, so I'm going to try to quickly sum up my conclusions:

Drop the class distinction between inferring and non-inferring nodes; everything provides an inference method, it just might emit a constraint failure without actually doing any work
Don't expect reduced trees to do anything useful in terms of typing
Restore type inference to values where applicable

None of this will address the problems I was having with the current round of updates, so I'll either need to back those changes out, layer more changes on top of the half-done stuff in there currently, or put the work into figuring out how to fix it the rest of the way.

In any case, it's late and I should get to bed.

Good night.

Diary 2025-07-24

2025-07-24T04:00:00-04:00

I'm trying to figure out what the right path forward is for this hobby coding stuff, but I used up all of my energy today on other things and I'm going to just go take things easy now. Maybe I'll be ready to work on the stuff I want to in, like, a few days.

Good night.

Coding 2025-07-23

2025-07-23T04:00:00-04:00

Ugh. The code I wrote for paths and constraints feels terrible to work with, actually.

I think there are two separate things I did wrong, and it's going to take me some time to figure out what I should be doing instead:

"I should make a path-like structure using __rtruediv__ on a base 'component' class to build things up."
"I'm bundling together nodes and context; I should write methods with narrowed self types on the bundled class that just forward to the method implementation on the wrapped node."

Both of these seem to get really unhappy in the context of my funky modular node types. The method forwarding is also a bit questionable from an implementation standpoint, because I kept finding myself reassembling the bundle that destructured itself as part of the method forwarding. Like, the stuff I thought was going to make things easier just kind of made it way fiddlier.

I've got a few things I'm trying out now, but I'm going to need more time than I have right now to really evaluate them, so I'm going to wrap things up for now.

Good night.

Weekly Roundup 2025-07-22

2025-07-22T04:00:00-04:00

Wednesday: I made some progress related to type checking. (To be honest, my writeup is too vague for me to remember.)
Thursday: I made a bit more progress, and identified some major stumbling blocks.
Friday: I got some stuff working. I mentioned "typing" in the writeup, and I'm now realizing that was ambiguous between what I meant and what would have been nice. I got the Python code to typecheck.
Saturday: I made something of a plan to move forward.
Sunday: Tired.
Monday: I tweaked the plan a little, and got the beginning of it taken care of; now it should just be a case of mopping up, with some minor new bits of code.

Next week, I'll try to get all of this truly squared away, and maybe work on some other things as well.

Coding 2025-07-21

2025-07-21T04:00:00-04:00

I've updated the base node classes for the System F implementation, thereby breaking everything else. I should just need to keep iterating on this until I fix all of the Mypy errors, but I'm not going to do that right now, since it's late and my focus is going elsewhere.

So, I'm not going to do any of this tonight, but here's what I think needs to happen:

Get the code to pass Mypy.
Get the tests to pass Mypy.
Get the tests to pass.
Take a break.
Audit the code organization; there is a lot of "this technically works, which is good enough for now".
Address test coverage gaps.
Fix the many outstanding lints.
Take another look at the documentation and try to make it nicer.

For now, I'm going to wind down and doodle.

Good night.

Diary 2025-07-20

2025-07-20T04:00:00-04:00

I'd been hoping to get stuff done with coding today, but it wasn't to be. I should have a chance tomorrow, which is very soon, so I won't have to wait too long to try things out. Anyway, I absolutely should not be up any longer.

Good night.

Coding 2025-07-19

2025-07-19T04:00:00-04:00

Okay, I made some progress, but it's getting late, so I'm just going to plan things for later, because I think I'm ready to update type checking. Here's how it should go:

Checking implementation for synthesizing comp terms needs to be updated to potentially generate a constraint; I need to figure out how to deal with the new "path" concept in this context.
Divide value types into synthesizing and checking; implement checking on synthesizing types.
Create a kind ascription type.
Write a checking implementation for comp types.
Actually update all of the method signatures.

At that point, I should get a giant pile of type errors out of Mypy. Once I grind through all of those, I'll get a whole bunch more type errors checking against the test suite, and I can update everything, and then hopefully the tests will finally pass.

For now, I'm going to get ready for bed.

Good night.

Coding 2025-07-18

2025-07-18T04:00:00-04:00

I threw together constraint implementations and got typing to kind of, sort of, work. I think it should be ready for me to try stuff out, but I want to get some rest before I give it a shot.

And I really tried to come up with something else to say, and failed. I'd better get ready for bed.

Good night.

Coding 2025-07-17

2025-07-17T04:00:00-04:00

I believe I have all of the scaffolding set up to rewrite the type checking code. There are a few shortcomings:

I haven't put together the majority of reporting-specific code.
It's possible I'm missing one specialized class, that fortunately is trivial to write if I need it.
The typing annotations are extremely rough, and I don't have a good intuition currently for how to shore them up.
Some of this code is just kind of messed up, and I don't feel like fixing it right now.
I need to decide whether constraints should work by explicitly calling a method, or coercing to bool.
Constraints need to capture the environment, or lots of things won't really work.

I think I need to get that last one squared away, and then I can chip away at the rest as I see fit.

Maybe I'd have gotten more done if I wasn't playing Gentoo Rescue and watching videos about games I don't play. Maybe not. Technically, we'll never know.

But I will try to get a bit more done tomorrow. For now, once again, bed.

Good night.

Coding 2025-07-16

2025-07-16T04:00:00-04:00

I've been messing with those code blocks from last time, and refining them. I'm feeling pretty good about the state of the code, but it's only going to come together once I start making actual constraints, to replace the minimal stub I've created.

Otherwise, I haven't thought too hard about this, because, again, Gentoo Rescue.

At this point, it's late and I need to get ready for bed.

Good night.

Weekly Roundup 2025-07-15

2025-07-15T04:00:00-04:00

Wednesday: I worked on generalizing the kinds of statements the type checker can make on failure.
Thursday: I had some ideas for representing those statements as data.
Friday: I preparation for this work, I wrote string representations of (most) types and terms.
Saturday: I finished off the string representations, for now.
Sunday: Zoned out.
Monday: I finally wrote some decent draft code of this stuff.

Next week, I'm going to refine the draft code into something nice and streamlined to use. Also, I'm going to look for other stuff to focus on as well.

Coding 2025-07-14

2025-07-14T04:00:00-04:00

I think if I'm going to get anywhere with this concept of a context, or a path, or whatever, I'm going to need to sketch things out before trying to create the classes. Let's do this as a before-and-after.

def _inferred_type(self) -> abc.CompType[TY | Type[TY, K], K | TypeKind]:
    func_type = self.rator.inferred_type()
    if not isinstance(func_type, Type):
        msg = f"Expected function, got {self.rator} of type {func_type}"
        raise TypeError(msg)
    self.rand.check_type(func_type.from_)
    return func_type.to

And the goal is something like:

def _inferred_type(self, path: Path) -> Generator[
    Constraint,
    None,
    tuple[
        abc.CompType[TY | Type[TY, K], K | TypeKind],
        Path,
    ],
]:
    func_type, type_path = (
        yield from Type.coerce(
            self.rator.inferred_type(
                path / AccessedFrom("rator")
            )
        )
    )
    yield from (
        self.rand.check_type(
            path / AccessedFrom("rand"),
            (
                func_type.from_,
                type_path / AccessedFrom("from_")
            ),
        )
    )
    return func_type.to, type_path / AccessedFrom("to")

This should give me something to chew on, so far as coming up with proper interfaces. I'm most interested in coming up with some way to cut down on the redundancy of generating the value and accessing the path. As I said, something to chew on. Anyway, I'm tired.

Good night.

Diary 2025-07-13

2025-07-13T04:00:00-04:00

I tried to rework stuff for Impliciula, and it didn't work out. I also did other stuff today, but ultimately I got frustrated because I couldn't figure out the right way to associate more data with types as part of checking.

Thinking about it, maybe I'll do better if I just try re-implementing the new system next to the old one, and see what I need to make it work.

But that's definitely not happening right now. Need sleep.

Good night.

Coding 2025-07-12

2025-07-12T04:00:00-04:00

I'm done with string representations for now, but I still managed to have kind of a rough day, so rather than necessarily trying to force implementing the new stuff I have planned tomorrow, I'm going to take things easy and see what I'm actually up for.

No more sentences, it would take too long.

Good night.

Coding 2025-07-11

2025-07-11T04:00:00-04:00

I'm working on stringifying everything nicely. I got a bunch of stuff done, including existentials, which are probably right, but confusing. I got to thunks, and then kind of zoned out for a few hours. Once I figure out the right representation of thunks wait never mind, I got it, now all that's left is handling universal types and the associated terms. That should be pretty quick once I've had some rest.

Anyway, I'm not going to stay up much later, so I'll call this entry here.

Good night.

Coding 2025-07-10

2025-07-10T04:00:00-04:00

I've worked out that probably the sensible formulation is to have some kind of "provenance" that I can bundle up with nodes, different classes for each kind of constraint that I've thought of, with a constructor that takes bundled nodes and operates on just the node data, to determine whether the constraint is violated. The problem is, I'm having trouble figuring out what the "provenance" should be, and what should be elided, and how.

I think part of the problem could be that I'm unsure whether making the AST more expressive would help. The problem there is that the plan has always been to compile down to this System F Omega stuff as a somewhat-intermediate representation, but until I have the higher-level data structures to compile down, the obvious way to construct these trees is going to be full-on artisanal.

In the interest of making things work, I'm going to try the following in the next few days:

Come up with a stub string representation for all computation terms.
Come up with proper string representations for all value terms.
Actually, see if I can somewhat present useful information for computation terms.
Create a union type to represent provenance.
Create a parametric bundling type.
Write the different constraint classes.
Try to give the resulting data a somewhat human-readable presentation.

For now, I'm going to be getting ready for bed as quickly as possible, because I got distracted by Gentoo Rescue.

Good night.

Coding 2025-07-09

2025-07-09T04:00:00-04:00

I've picked out seven basic structures of statement about relations between values and types, etc, in the System F code. Once I've generalized those to kinds and computations, I should have all of the planning I need done to get back into the code. No promises, though.

I was once again focusing on other things today and sort of taking it easy, so I once again did that work in the space of a few minutes. But I've got that information, and I can convert it into a checklist and start working through it. Hopefully, I can get on that tomorrow, but if not, oh well.

Anyway, I want to get to bed now.

Good night.

Weekly Roundup 2025-07-08

2025-07-08T04:00:00-04:00

Wednesday: Oh no, bidirectional type checking is giving me different error messages when I check malformed syntax trees. Better think about this for over a week!
Thursday: I came up with a plan/excuse for how emitting error messages is kind of like generating a constraint set.
Friday: Felt bad, but had some ideas for how to handle these error messages in a more principled way than "it's strings!"
Saturday: I tried to put together a list of everything that can go wrong for one of these syntax trees.
Sunday: I took that list and tried to converge the phrasing so I can have half-a-dozen classes/helpers, instead of, like, twenty.
Monday: "Hm, some of these messages kind of have to glom onto subsequent messages; I need to model that somehow."

Next week, I'm going to take my message drafts and try to put them through a few rounds of prototyping.

Coding 2025-07-07

2025-07-07T04:00:00-04:00

Okay, I've drafted constraint descriptions, and I've got some ideas for creating classes based on common patterns in phrasing. Once I'm feeling a little better, I'll hopefully have something prototyped to see whether I'm going in the right direction.

I've got some idea that I'd like a kind of constraint description that just affixes onto a more specific one. Not sure if it should be prefix or suffix. I'll think about that while I wind down.

Good night.

Coding 2025-07-06

2025-07-06T04:00:00-04:00

Okay, here's the tack I'm taking. I'm translating each of the possible constraints I developed into a short statement of that constraint, with placeholder values to be filled in. From these, I intend to create classes representing the different constraints.

The hope is that I can figure out some way to chain these constraints together to form some sort of causal narrative. We'll see how that works out. I kind of need sleep before I can make any meaningful progress.

Good night.

Coding 2025-07-05

2025-07-05T04:00:00-04:00

I just put together a preliminary list of everything that can generate an error when checking/synthesizing types. The next tasks are to figure out how and whether these all interact, and where to carry out some of the checks.

After that I can work on stuff like building up error contexts. I'd like to work on that now, but it's late and my focus is shot, so I'm going to try to get some rest and see if that addresses the various problems I'm having.

Good night.

Coding 2025-07-04

2025-07-04T04:00:00-04:00

Oh wow. On the one hand, I've made some progress planning things, but on the other hand I feel awful. I'm going to try to quickly lay out a plan, and then get to sleep and hope my body recovers from whatever is happening.

Long term, I want to be dealing with some kind of error object instead of strings, and such a thing can probably be broken down into two basic categories: triples of context, expected type, and actual type, and triples of context, expected kind, and actual kind. Long term, I want to be yielding these from a generator, but for now, I can get tests working again if I simply use them to generate an error message. I think I want the "context" templates to be available at the top level, so I can write tests for them withouth having to synthesize awkward syntax trees, but on the other hand, I want to make sure incorrect trees generate the right error messages, so I don't know, I'll have to think about it more.

For now, I'm going to try to read and let my brain spin down a little.

Good night.

Coding 2025-07-03

2025-07-03T04:00:00-04:00

I'm still spitballing ideas for reworking error generation. One thing that I'm kind of stuck on is that I'm fairly sure I'll want to do typing via constraint solving eventually, but do I want do try to get it working right now? And if I don't do constraint solving right now, can I find some way to minimize the disruption of switching to something now, and then from that thing later?

Alternatively, can the ideas I'm messing with currently grow to handle the stuff I think I'll need constraint solving for later? Well, let's consider the "later". I'm going to want to check function bodies "in isolation". In other words, a given function's type ascription checks inward, and it can infer outward from any other ascription, (which points me towards some mild problems with how I'm doing bidirectional typing, and which may be the problem the paper was trying to warn me about), and I'm going to want to verify that the predicates from the refinement types hold. Verifying this may involve the invocation of lemmas, and testing the subtype relation based on predicate implication.

My gut and my heart tell me that this can be made to work in a syntax-directed fashion, but unless I can (figure out how to) prove this, I at least need a backup plan of some kind. Okay, I see "Refinement types combine SMT decidable constraints with a compositional, syntax-directed type system", but do I gotta bring in SMT? Maybe I could conceptualize the current "error" plan as only emitting unsatisfiable constraints? Error reporting would then notionally involve, for each constraint, finding the smallest set of other constraints that are satisfiable without it, but unsatisfiable with it, for some definition of "smallest", and possibly "the".

With all of that in mind, I've got a fairly good idea of how I can proceed with this idea, but I want to take things slow, so I'll wrap this up for tonight and see if I feel ready to take it on tomorrow.

Good night.

Coding 2025-07-02

2025-07-02T04:00:00-04:00

Okay, here's the deal. I think I see how to recover something like the error reporting functionality I had in pure inference mode. The current type-checking code is incapable of reconstructing the context available to the caller, which is where the error used to be generated from. So, if I want to replicate the same error information, then I need to build up some form of context object at the outermost layer of checking, and propagate it towards the leaf nodes. From inspecting rg output on my codebase, I can see that no method ever performs more than one check; the unimplemented nodes are solely types, so that won't change anything. However, I do kind of want to try making kinding bidirectional for the heck of it, which will result in type checking sometimes turning into kind checking, and will lead to some form of branching in the error message generation. Currently, checking failure leads to an exception being raised, but I want to have the way open to generating typing constraints instead, which is going to be a Whole Thing, and means that every validation path will end up being taken...

Convincing Python to suddenly be Prolog sounds like a project all its own. The feeling I'm getting thinking about this is that I'm going to need to do some experimentation to figure out what this code should look like, and how many forms it's going to have to go through.

One thought that comes to mind is that replicating the current behavior doesn't require an exception to be raised in a check function, only in the inference function that called it. So, perhaps I could rework the check functions to return some form of problem report...

Perhaps what is needed is some idea of how more featureful compilers report errors. I'm going to need to track down some more papers, I guess. And get back to taking notes.

Anyway, I need to get ready for bed.

Good night.

Weekly Roundup 2025-07-01

2025-07-01T04:00:00-04:00

Wednesday: I planned the layout of the cover for Way Too Fast!!
Thursday: I carried out some of those plans, and gave up on others.
Friday: I finished the cover, put together the PDF, and had [FUN] figuring out Itch.io's jam submission modal, which has what I would characterize as some rough edges.
Saturday: I got back to Impliciula, tried to simplify the code, and failed, so I scaled back the planned simplification.
Sunday: The planned simplification worked, so I planned further changes.
Monday: I made those changes, to introduce bidirectional typing, and ran into a few issues.

Next week, I'm going to try to address those issues, and maybe work on a few other things.

Coding 2025-06-30

2025-06-30T04:00:00-04:00

I've got bidirectional typing set up. Most things are still doing inference, though. Functionally, what I have is "functions require ascriptions, and one of the tests is broken because I messed up error reporting". The former is a step forward, I think, and the latter... I'm going to need to ponder hard how to fix this. (It's possible that I'll try switching some things from synthesizing to checking, but right now the only reason I could think of to do so is if it would somehow improve error reporting.)

I'll see what I come up with, but for now I'm going to take it easy.

Good night.

Coding 2025-06-29

2025-06-29T04:00:00-04:00

Okay, after a good but exhausting day, I've successfully culled the type variables in the trees down to "term, type, and kind". Now, the different sorts of term and type that I had before are distinguished by the specific class that they inherit from, not which type variable they are associated with. This is a step forward, because now I can introduce more base classes into the node hierarchy without multiplying the number of type variables in play.

And I want to do that because I'm investigating bidirectional typing, which introduces a distinction in which all nodes can check types, but some nodes can also synthesize types, and I want to try encoding that distinction into the node types, such that, if a particular node type needs one of its children to synthesize a type, then that child will be able to synthesize, if a valid type exists. In other words, the Python type system should be able to guarantee that all necessary type ascriptions are carried out.

Now, one of the reasons I want to explore bidirectional typing is that I suspect it will improve some of the ergonomics around function typing; this justification is kind of just there to make me feel better about totally redoing the typing system in a way that will also break most of my tests.

It's late, and I don't currently have the energy to evaluate two important questions:

Have I picked the right directions for everything, or should I make some fairly obvious tweaks?
What do I want the new interface to look like? I'm currently leaning toward an "inferred_type" property and a "check_type" method.
The secret third question: is it possible to guarantee that a synthesizing node always reduces to a synthesizing node, and if so, what compromises are required? (I mean, a synthesizing node knows its own type, by definition, so I guess it could just generate a possibly-redundant ascription node? Or there could be methods on the relevant node types to say "given this type, return some node that has the same behavior, and synthesizes the type, if it synthesizes at all".)

Anyway, that's a lot to think about, and there's no way I'm making any good decisions on it in the next half hour, so I'm going to wrap things up.

Good night.

Coding 2025-06-28

2025-06-28T04:00:00-04:00

All right, I'm back to coding, and for my triumphant return, I... discover that my planned simplification to the node types simply does not work. Fortunately for the code, there is a less ambitious change that should work, and still accomplish the improvement that I'm trying to get.

Basically, before, I was trying to collapse every node type variable into one, and it turns out that going that far breaks it, but I should be able to collapse the node type variables into kind, type, and term, and then freely subclass from those base classes. I'll see if that all makes sense to me tomorrow, but I've got a good feeling about this new plan.

Anyway, I really should get ready for bed now.

Good night.

WTFJam 2025-06-27

2025-06-27T04:00:00-04:00

The cover is finalized, the PDFs are generated and combined, the document is apparently not C5 envelope sized but I have no idea what it is, but close enough. Everything is done.

Except you haven't submitted it.

Itch? Itch?! What is the problem?

Some time has passed since the previous line, and we now understand everything.

Heeheehee, I missed the "submission" badge on the project page and the "submit" modal never told me that I already submitted something.

I sleep now.

Good night.

WTFJam 2025-06-26

2025-06-26T04:00:00-04:00

I've scaled back my ambitions for the cover of Way Too Fast!!, and taken a few hours to get more or less what said scaled-back ambitions call for. My plan now is to sleep on this, and see how I feel about it tomorrow. There is one tweak I might attempt, but I have to be willing to bail if it doesn't work out.

Regardless, I hope to finalize the cover design tomorrow, do one more pass over the playset contents, and submit this, then be done with it.

Anyway, I should get ready for bed now.

Good night.

WTFJam 2025-06-25

2025-06-25T04:00:00-04:00

All right, I've sketched out all of the major elements of the Way Too Fast!! cover. My plan for the next few days is to recreate my sketches as vector drawings, and hope they come together somewhat nicely. I don't want to try to put that together right now, so I'll just note that it looks like the template I'm using is meant to be printed at a size of C5 Envelope, or 6.38 by 9.01 inches. So, if I get the cover art to match that, hopefully it will all look good.

I will try to see tomorrow whether it is feasible for me to put all of this together myself; if not, I'll probably just go for something rougher.

Anyway, I let it get late, so let's wrap this up.

Good night.

Weekly Roundup 2025-06-24

2025-06-24T04:00:00-04:00

Wednesday: I got started on objects.
Thursday: I got everything drafted, and decided to let things sit for a few days.
Friday: I started messing with Impliciula again, and complained about the things I decided I needed to do.
Saturday: I got done with the stuff I needed to do to Impliciula, but decided to hold off a few days before doing the stuff I want to do.
Sunday: I tried to take things easy and do some basic, undemanding tasks.
Monday: I got those basic tasks done, while suffering.

Next week, I'm going to try to make a cover image for Way Too Fast!!, and get it in by the weekend.

WTFJam 2025-06-23

2025-06-23T04:00:00-04:00

I've got most of the draft for Way Too Fast!! loaded into a (slightly janky) template, and now there are just two major things to work on:

Picking everything out for the quick setup recommendations
Making a snazzy cover image

I'm not totally sure I can put together what I have in mind for the cover, but I'll try for as long as I have time.

For now, I had a rough day, and I need to wind down.

Good night.

Diary 2025-06-22

2025-06-22T04:00:00-04:00

I made some light edits to Way Too Fast!!, and I plan to start getting the draft into the template soon.

I'm making sure I'm comfortable with the next step to update Impliciula, and I've been rereading various papers to get some idea of the task that I've set for myself to incorporate the advanced features that are kind of the point of all of this.

Anyway, it's late and I need to get ready for bed.

Good night.

Coding 2025-06-21

2025-06-21T04:00:00-04:00

I've managed to write tests to cover all of the code that I want to cover right now. Rather than jumping right into refactoring the type annotations, I'm instead rehashing the problem and my intended solution, to try to be sure that I won't somehow make things worse.

My plan right now is to get back to Way Too Fast!! in some capacity tomorrow, and work on this code as I see fit. For now, I need to get ready for bed.

Good night.

Coding 2025-06-20

2025-06-20T04:00:00-04:00

I was pretty tired today, so I ultimately ended up poking at Impliciula's code a little, and making some improvement to test coverage. At this point, I believe the right thing to do is to make careful improvements to the method coverage for universal types.

But what I want to do is make some code changes that I haven't tested, but I'm pretty sure will work and dramatically cut down on the verbosity of the types I'm using to implement this. That will then open up the potential to define more node types, which will enable me to totally rework type inference, in a way that's going to require updates to all of my tests...

But no, the responsible thing to do is to get some form of coverage around stuff like weird nestings of type functions. Ugh.

Anyway, I spent long enough sloooowly typing this, I'd better wrap up and be ready for bed.

Good night.

WTFJam 2025-06-19

2025-06-19T04:00:00-04:00

I've got all 144 entries for Way Too Fast!! drafted, so I'm letting that sit for a day or two before I do an editing pass on it. (Can you believe I thought I might be able to get two games done for this? Given the scale of the game I didn't do, a ludicrous idea.)

Anyway, I got back into working on Impliciula, and I've got to work out a better way to generate test cases, because what I have now? Sucks.

Anyway, let's wind down now.

Good night.

WTFJam 2025-06-18

2025-06-18T04:00:00-04:00

I'm through with locations, and now I need to make more progress on objects. I made a little progress today, but I need 26 more. It shouldn't be a big deal, but I don't have the focus for it right now.

I'm going to wrap this up for tonight and read stuff.

Good night.

Weekly Roundup 2025-06-17

2025-06-17T04:00:00-04:00

Wednesday: I got the relationships for Way Too Fast!! almost wrapped up.
Thursday: I finished up the relationships, and started on the needs.
Friday: I got to halfway through the needs.
Saturday: I finished the needs.
Sunday: I didn't have much to talk about.
Monday: I got to halfway through locations.

Next week, I'll try to wrap up locations, and work on items. Hopefully that gets all done soon.

WTFJam 2025-06-16

2025-06-16T04:00:00-04:00

I think I'm about halfway done with locations. If I keep the pace, I'll have everything drafted by the end of the week. The big question is, once I have a draft, do I just send it to get the jam over with, or do I actually try to polish it any before publishing?

The decision I make may end up influenced by the fact that I'm itching to get back to futzing with the code that's supposed to eventually become Impliciula.

Anyway, I should wrap up for now.

Good night.

Diary 2025-06-15

2025-06-15T04:00:00-04:00

As I anticipated, I didn't have much time to work on Way Too Fast!! today, and I don't feel like talking about what I did do.

I think I need to get away from this screen and zone out for a bit, so let's see about that.

Good night.

WTFJam 2025-06-14

2025-06-14T04:00:00-04:00

Needs are drafted. I'm kind of assuming tomorrow will be a wash, and I'm not too sure about the day after that, but my hope/assumption is that I can get something together for locations and objects next week.

We shall see.

For now, I should wrap up and be ready for tomorrow.

Good night.

WTFJam 2025-06-13

2025-06-13T04:00:00-04:00

Halfway through needs, and I need access to my notes to get much further, but it's late and I don't really want to get them right now.

I'll try to wrap up needs tomorrow, or by the end of the weekend at the latest, then get to work in earnest.

I'm going to wrap things up for now.

Good night.

WTFJam 2025-06-12

2025-06-12T04:00:00-04:00

The relationships table is now Good Enough. I'm now working on coming up with some kind of categorization scheme for the draft needs, but it's slow going; I've currently got ten in two categories, and of course, I need thirty-six in six.

Right now, I'm banking on the hope that I'll be able to grind through objects and locations really quickly, and it makes sense to focus on needs, in order to shore up the foundation. We shall see.

Anyway, I'm out of focus for this for now, so I'll try to get some serious progress on this tomorrow. For now, I've got to wind down.

Good night.

WTFJam 2025-06-11

2025-06-11T04:00:00-04:00

I've assigned all of the draft relationships for Way Too Fast!! into sub-categories, and filled in most of the blanks. There are just a few blank entries left, and a few that I'd like to come up with something different for, and one sub-category that doesn't really make sense yet. But just the one, so if I can square all of this away, that's relationships handled.

Needs is going to be interesting, because I don't yet have a good grasp on what kind of sub-categories will fit the draft needs.

Anyway, I tried to round out the relationships, but I'm not quite there yet. Just a little more, but not right now; it's late.

Good night.

Weekly Roundup 2025-06-10

2025-06-10T04:00:00-04:00

Wednesday: I had a few more thoughts on Impliciula.
Thursday: I did some very basic planning for WTFJam.
Friday: I somewhat acted on those plans.
Saturday: I didn't get anything done that I wrote down.
Sunday: I got to about a quarter done.
Monday: I made some more progress.

Next week, I'm going to try to finish up relationships and needs, then work on locations and objects. Whenever those are done, I'm unfortunately going to need to give relationships and needs a few more passes, because there are... issues.

WTFJam 2025-06-09

2025-06-09T04:00:00-04:00

I'm still super tired from yesterday, but we did discuss the core stuff for Way Too Fast!! earlier. (I also took a bunch of notes for Impliciula, which, if I'm going to do that, I guess I'm going to do that, but, like, the jam. Got to jam.) I'm actually getting this post written early, so I can try and get a bit more done tonight.

And I spaced out for a bit to see if I'd come up with anything else to post, and I didn't. So we're done for now.

Good night.

WTFJam 2025-06-08

2025-06-08T04:00:00-04:00

We had plans for most of today, so I ended up not getting too much done, but I have gotten to approximately the halfway point on relationships and needs. If I keep up this "pace", then I should just barely have something to submit by the end of the month; I'd prefer to work a little faster. We'll see how things work out tomorrow. For now, I want to go lie down.

Good night.

Diary 2025-06-07

2025-06-07T04:00:00-04:00

I was out of it today. It was so hot...

Let's see what I can accomplish tomorrow, or the day after that.

Good night.

WTFJam 2025-06-06

2025-06-06T04:00:00-04:00

I put in the effort today to come up with relationships and needs at what felt like a decent pace. I still don't have thirty-six of each, though, so I'm going to have to put in more focused effort over the next few days.

For now, I'm going to wrap up and try to deal with how dang hot it is.

Good night.

WTFJam 2025-06-05

2025-06-05T04:00:00-04:00

Okay, here's the idea. I'm going to put together a Fiasco playset (first edition, since I never looked into second edition) about racing, as depicted in Speed Racer, perhaps Wacky Races, probably other things.

To get it together, I'm going to need to come up with 144 different relationships, needs, locations, and objects. Getting the relationships and needs right will determine whether I can get this to work, and the locations and objects will help convey the vibe. I'm going to need to come up with several of these a day to be done within a month, and I think I'm currently not on that pace, so I'm going to try to get more done in the coming days.

For now, I need to wind down.

Good night.

Coding 2025-06-04

2025-06-04T04:00:00-04:00

I don't want to write up my WTFJam progress because there isn't much yet. My goal is to come up with 144 entries for a Fiasco playset, and I've made some piecemeal progress on that, but I'm going to need to come up with many of these things a day to actually finish this in June.

I ended up devoting more thought to subtyping in Impliciula. The basic implementation of record subtyping shouldn't present too many theoretical hurdles, and I'm reading up on refinement types, but the crux of all of this is going to be the effect system. The crucial interaction that will make programming in Impliciula the way I want to, like, even possible, will be that refining the arguments of a function to a subtype will sometimes make it possible to promote the function type to a supertype that lacks specific effects.

(Actually, it occurs to me that function inlining could make record subtyping more tractable under some circumstances, but, oof, that feels like something that would have to be opt-in.)

Anyway, I need to wrap up for today.

Good night.

Weekly Roundup 2025-06-03

2025-06-03T04:00:00-04:00

Wednesday: I had ideas about how to improve the Impliciula prototype.
Thursday: I thought I was ready to implement type operators.
Friday: But instead the refactors continued.
Saturday: The refactors continued, and revealed a weird sharp edge in Mypy.
Sunday: I looked over the current coverage deficit.
Monday: I fixed up some areas of the code, and improved the typing situation.

Next week, I'm going to try to stop touching the code, and focus instead on planning next steps, and working on other projects.

Coding 2025-06-02

2025-06-02T04:00:00-04:00

I found a paper on call-by-push-value that laid out how existential types should be set up, so I partially followed that, and this simplified the code a lot. I'm still not totally vibing with the idea of type variables standing in for free-standing computations; I skimmed the paper, so I have no idea whether they did that out of a sense of symmetry, theoretical interest, or if sometimes you really do just have to parameterize by the sort of computation you're doing.

Anyway, I figured out a major source of friction in modeling these recursive structures, so now I've got a weird template to follow for defining future nodes. It would be interesting to jump right into doing that, but I want to at least try to increase the coverage and document the code first. For documenting the code, I'm looking into embedding LaTeX, because honestly, I kind of don't want to use words to explain what these methods are doing.

However, I feel like I should take a bit of a break from this, because I've been polishing this code really hard. So, I'm going to try to put together an entry or two for WTFJam 2025.

But for right now, I need to get ready for bed.

Good night.

Coding 2025-06-01

2025-06-01T04:00:00-04:00

After a bunch of messing around, I've made some coverage improvements. At this point, the lack of coverage is localized to five files, and can be traced to three basic areas:

Kind checks, which will require higher-kinded types to properly trigger.
I haven't exercised a lot of the code I've written for universal types.
I just don't have a good intuition for how to relate "how existential types are implemented" to "the stuff I find myself wanting to do with existential types", so I'm a little scared to touch them.

Hopefully, I can deal with the latter two soon. I may need to take a break from this project in the near future, but we'll see.

Good night.

Coding 2025-05-31

2025-05-31T04:00:00-04:00

Oh boy, I sure was messing with this stuff today. I got the nodes module filled up, then broken (mostly) apart. Along the way, I found a really nasty condition in Mypy, that I assume everyone else's code is too normal to trigger.

My plan at this point is to investigate just how much of the uncovered code, can easily become covered code.

I'll see how that goes tomorrow.

Good night.

Coding 2025-05-30

2025-05-30T04:00:00-04:00

You might not think that I could keep on coming up with massive refactors, but you would be wrong. My goal currently is to move the method implementations back to the nodes, and then break up the nodes module because it'll be over a thousand lines at that point, which is way too much. What makes this state of affairs different than what came before is, I'm encoding more details about tree structure onto the nodes, so now, fingers crossed, I can statically... be sort-of-confident that I won't have stuff from higher levels "bleeding down" into lower levels, in terms of tree constituents during evaluation.

I need to wind down for now, though. I'll be messing with this more tomorrow.

Good night.

Coding 2025-05-29

2025-05-29T04:00:00-04:00

Okay, I need to let the code base sit, at least for tonight. I believe I've worked through all of the refactors I'm going to manage for now, and laid the groundwork for the next phase of all of this, implementing type operators.

After that, I can either rush into adding row polymorphism, or I can focus on raising the coverage, which would be the responsible course of action.

...

I'll try to be responsible. And that starts with getting to bed.

Good night.

Coding 2025-05-28

2025-05-28T04:00:00-04:00

I had a bunch of thoughts about where to go next on Impliciula, but then I had some realizations about better ways to accomplish stuff currently, and so right now I'm rewriting various parts of it in preparation for those new ideas, and so it's just kind of a mess at the moment and I need to go sleep.

Good night.

Weekly Roundup 2025-05-27

2025-05-27T04:00:00-04:00

Wednesday: I was experiencing some difficulties implementing call-by-push-value.
Thursday: I started some pretty extreme refactors to the Impliciula prototype code base.
Friday: I tried to explain those refactors.
Saturday: Things seemed to be working.
Sunday: Things seemed to be working, but in a kind of unpleasant way.
Monday: I got distracted from shoring up my test coverage by the siren song of more features.

Next week, I'm going to try to finish up the latest bout of feature nonsense, then actually write tests that are still based around directly manipulating syntax trees.

Coding 2025-05-26

2025-05-26T04:00:00-04:00

So, I was totally planning to improve the test coverage for System F. But instead, it seems I've added existential types. With no tests. And it's incredibly janky because I wasn't sure how to adapt it to call-by-push-value, and I figured I'd rather muddle through than look for a paper. And I tried to pare back the bloated size of the different modules, and as a result I'm probably concealing many errors from Mypy by accident.

I'm fairly sure this is recoverable. I just need to go back to the basics of System F, and start testing. Tomorrow.

Good night.

Coding 2025-05-25

2025-05-25T04:00:00-04:00

Okay, I've got the System F prototype spun up and ready for testing, but it took me all day and the modules are hundreds of lines long. (I didn't think to check the lengths before the five-or-so refactors necessary to reach this point.) Hopefully extending System F will be quicker, when I'm ready for it, now that I've got some groundwork laid, but that remains to be seen, and I do not want to rush into that.

For the near term, I'm going to have to find something a little less intensive to work on. For the very near term, bed.

Good night.

Coding 2025-05-24

2025-05-24T04:00:00-04:00

Going to have to make this quick. I'm in the middle of building up the System F implementation, but there are so many tiny methods to write. I basically need to implement the evaluation rules for the lambda calculus again, but with types instead of values, and so this means that, not only do I need to write more implementations per method, but I need to write more methods overall.

Where I stand with this is:

I've proved out the general idea of using singledispatchmethod and classmethod for holding the implementation, although I haven't yet tried to deduplicate the implementations between modules.
I'm midway through writing these methods, but there's a bunch more to be done.
I need to figure out some way to organize the class body; as far as "flat is better than nested" goes, this is going way too far. Hopefully the deduplication efforts will make things behave somewhat better, but I can't try that until I figure out what it will look like, which is maybe a better use of my time right now than grinding through more method definitions.

Anyway, I need to wrap things up.

Good night.

Coding 2025-05-23

2025-05-23T04:00:00-04:00

Okay, here's a little more detail on the state of things.

My goal was to be able to represent, say, both the simply-typed lambda calculus, and System F, with as much code sharing as I can. In particular, there is a lot of common structure between them, and a lot of common evaluation logic. But the differences between them are problematic if they're represented in "normal-looking" code. Because System F has more kinds of node than the simply-typed lambda calculus, not all valid System F expressions are valid expressions in the simply-typed lambda calculus. If we figure that the expressions of simply-typed lambda calculus would make up a subset of the expressions of System F, then that means that the representation of System F must be a superclass of the representation of the simply-typed lambda calculus? But removing node types by subclassing is counter-intuitive, and if the evaluation logic is represented by methods (normal and obvious object-oriented practice), then subclassing in either direction introduces Liskov/variance issues.

So, we're not going to be able to reuse that common evaluation logic or structure through subclassing. What then?

I'll be honest, this next part's a bit of a haze, and I went down some weird dead ends, so let's skip ahead a little.

Due to various paper-cuts, I ended up not trusting the individual nodes with any logic outside of, like, alternative constructors, attrs validators, and stringification. Everything else, I converted from properties and methods to a bunch of singledispatch functions that all call each other recursively and interact with some global constants and functions. So, this delivers the same functionality that I had before, and has made all of the node types lean, at least. (Although, note that I've lost some type-checking robustness because it's not checking that all necessary types are registered. It would be good to find some way to guarantee that...) But how to allow for trees made up of different types?

Well, here's my plan, which I haven't tried yet, but I'm pretty sure will work:

I can define recursive type aliases, so what if I replace every union of node types in a node type definition, with a type variable? Then, to restore the original tree structure, I create type aliases to the filled-in versions of the nodes, then union those aliases together to get the parameter to fill into the generic version. To add more nodes, I simply add more types to the union.

But now let's consider the evaluation code. Sure, some evaluation code is now reusable, like determining the type information associated with a constant node, but most of these singledispatch functions refer back to each other in a recursive manner, that makes them unsuitable for reuse. If only we could somehow defer the reference, via some form of late binding...

It's a class. I'm taking all of these singledispatch functions and global constants, and sticking them in a class and turning them into singledispatchmethod classmethods.

Now, it'd be really neat if subclassing were practical now, but it is, for a few reasons at different theoretical levels, not. Instead, my current plan is to rewrite these functions to be generic, and then have the singledispatch machinery fill in the concrete types. (I mean, assuming Mypy cares. At runtime, this stuff is just all going to vibe.) Then, I can just call the register methods inline instead of as decorators. (I feel like this whole thing should work, but I'm also acutely aware that Mypy's singledispatchmethod support has (or, hopefully, had) some weird and confusing sharp edges and holes.)

Anyway, the basic idea for reuse is to define Protocols that lay out the methods and attributes that must exist on the concrete classes, parameterized of course. Then, at long last, we actually have a situation in this project where subclassing is appropriate.

Hopefully, I'll have the energy tomorrow to properly try this, but until then, I've at least got this stuff outlined.

Good night.

Coding 2025-05-22

2025-05-22T04:00:00-04:00

Aside from some gaps in mutation coverage, I believe the code is in a good, if transitional, state. Basically, I got the tests to a good state (aside from some gaps in mutation coverage), and then I refactored things so that there is no behavior on syntax nodes, and instead all the behavior lives in singledispatch functions. The goal here is to now parameterize the node types, and then only have the singledispatch functions be defined for specific type instances.

My hope is that I can then reuse individual function implementations by making the signatures just permissive enough that I can register them in multiple places.

If this works, it'll be great. If it doesn't, then I just kind of exploded my code for no real benefit.

You'll have to wait and see, because I'm definitely not trying anything else tonight.

Good night.

Coding 2025-05-21

2025-05-21T04:00:00-04:00

Just one more class bro. Just one more class, and I'll have a correct implementation of call-by-push-value.

Okay, technically, it's more that I implemented the class probably correctly, but I haven't tested this, but I'm just... done for now.

I'd like to believe there's a light at the end of the tunnel, but what I see of that light looks, to be frank, pretty weird. For one thing, I'm still not sure how I'm going to handle creating and translating between these highly-similar languages. For another, my goal with implementing call-by-push-value was to have a distinction between "pure" and "impure" computation ready to go when I manage to reach 1ML, but I don't understand the different parts of all of this well enough to grasp whether the FuncType | ReturnType distinction I have manages to actually capture that.

At least it hasn't been too hard to keep mutation coverage up.

Anyway, I have nothing more to say, so I'm getting ready for bed.

Good night.

Weekly Roundup 2025-05-20

2025-05-20T04:00:00-04:00

Wednesday: I was tired and sickly.
Thursday: I decided to try working on Impliciula.
Friday: I made some progress on the prototype, and thought about taking it beyond a prototype.
Saturday: I put together a lot of the scaffolding for working on Impliciula.
Sunday: I worked on the actual code, and also more scaffolding.
Monday: I maxed out the various metrics I'm taking, and pondered how to elaborate up to System F.

Next week, I'm going to be taking a little detour, because I decided to clarify the calling semantics of my simply-typed lambda calculus implementation, and I know without checking that the changes I made utterly cooked my test suite.

Coding 2025-05-19

2025-05-19T04:00:00-04:00

Okay, I got the simply-typed lambda calculus code up to snuff in terms of coverage and mutation coverage. Unfortunately, I'm not ready to elaborate up to System F, because I'm not totally sure how I'd like to.

See, the issue is that the elaborations, at least to start with, have lots of similar structure, just with More Stuff, and the way to fit the More Stuff in is... I think I see how I want to do it, but there are some complications to address. Because suddenly it's possible to give expressions new kinds of types, and have new kinds of expression. So, how to accommodate these extensions? It sure seems like the way forward is to parameterize the types and expressions that are allowed, which may push Mypy's support for recursive type expressions to the limit. Something like:

type Type[T] = BaseType | FuncType[T] | T
type Expression[T, E] = Value[T, E] | Abstraction[T, E] | Application[T, E] | E
type STLCType = Type[Union[()]]
type STLCExpression = Expression[Union[()], Constant[Union[()], STLCExpression] | Operation[Union[()], STLCExpression]]
or maybe type STLCExpression[T, E] = Expression[T, Constant[T, STLCExpression[T, E]] | Operation[T, STLCExpression[T, E]]]

I'll have to think about this and experiment with it a bit. Hopefully at a time when the internet isn't going up and down like a yoyo.

Anyway, that's more than enough time writing, so let's wrap this up.

Good night.

Coding 2025-05-18

2025-05-18T04:00:00-04:00

Okay, let's see if I can go over this very quickly.

I validated that Poodle was working, extended the scope of the ruff checks, set up pydoclint, activated a bunch of Ruff rules, deactivated a few Ruff rules, made a module for tracking typing contexts/environments, got the documentation from "not really working" to "sort of working", and started copying over code from the prototype. Oh, and I reorganized some minor bits of the repository a few times.

The current slog I'm on of documenting the prototype is forcing me to revisit questions like "is there any point to writing a Returns section for a property, is that not what the summary should be telling you?"

In any case, now I've got something to work on, even if it kind of... sucks, and we'll see how things work out when I finally have enough stuff together to elaborate this from the simply-typed lambda calculus up to system f.

Anyway, I really need to get to bed now.

Good night.

Coding 2025-05-17

2025-05-17T04:00:00-04:00

I set up a repository for developing a Python-based interpreter for Impliciula. I've got it set up with tox, running tests with pytest, with and without coverage, linting with ruff, doing type analysis with mypy, mutation testing with poodle, and building docs with sphinx. At this point, all I need to do is write the code.

Good night.

Coding 2025-05-16

2025-05-16T04:00:00-04:00

I've got a basic prototype of custom operations implemented. What is left...

Move this code into a project so I can package and better organize it.
Write tests.
Run the tests automatically.
Refresh my memory about how to get around the lambda cube, and to go beyond it.

My hope is that that last bit won't be too hard to do, now that I have a baseline of "this is how we're handling the lambda calculus".

Anyway, better wrap things up.

Good night.

Coding 2025-05-15

2025-05-15T04:00:00-04:00

All right, let's see what I did. In between the last time I wrote about Impliciula and now, I concluded that I was making things too hard on myself by trying to target LLVM off the bat. As such, I'm going to be focusing on implementing an interpreter for the simply-typed lambda calculus, and try to elaborate that until it can handle Impliciula code.

Now, it seems I'd sort of started on this already, so I mainly focused on cleaning up and extending the Python code that I already had lying around. At this point, it probably works, but I'm pretty sure I need to figure out stuff like, some way of defining primitive operations. Once I have that, I should be able to start elaborating.

Probably something with Protocols or ABCs. I'll think about it tomorrow.

Good night.

Diary 2025-05-14

2025-05-14T04:00:00-04:00

I'm still not ready to get back into my "main" projects. Trying and failing to take things easy.

I'll just leave things here for now.

Good night.

Weekly Roundup 2025-05-13

2025-05-13T04:00:00-04:00

Wednesday: I made some progress on the conlanging stuff.
Thursday: I explained part of what I'm trying to do with that code, but I assume it didn't really make any sense.
Friday: I thought some about where to go next with all of that.
Saturday: But then I was tired.
Sunday: And I still sort of had a migraine.
Monday: And then I worked on private wiki stuff instead.

Next week, I dunno. I've got stuff I want to work on, but also I am pretty tired...

Diary 2025-05-12

2025-05-12T04:00:00-04:00

Hm. Still not much to write up.

I'm drafting stuff for a wiki on Clifford worldbuilding stuff. I'm not sure when I'll feel like it's ready to put out there, but hopefully I'll be ready before it's "done".

Anyway, I can try to get a bunch more written right now, but I really shouldn't, so let's call it here for tonight.

Good night.

Diary 2025-05-11

2025-05-11T04:00:00-04:00

I got nothing that I want to write up done today, because it turns out I've been still recovering from that migraine, and I'm kind of sensitive to light. Which makes it kind of a problem to try to use my laptop.

We'll see how things shake out over the next few days, but I think I just have to accept the possibility that I'm not going to be doing much.

Good night.

Diary 2025-05-10

2025-05-10T04:00:00-04:00

I think I did stuff today, but I was too tired to work on what I wanted to, or to recount whatever it was I did. I'm just going to see about wrapping things up nicely for myself.

Good night.

Coding 2025-05-09

2025-05-09T04:00:00-04:00

Okay, the prototype code isn't any more organized, but I am closing in on documenting some of the functionality that I haven't even tried to implement yet. There are a few directions I can take this next, once I'm feeling a little less under-the-weather.

I can try to improve the organization and navigation of the notebook that I'm prototyping this all in.
I can start moving things into modules and importing them.
I can focus on improving the task runner capabilities for the as-yet barely touched original repo, so I can have assurance of test quality.

Anyway, I really need to rest my eyes, so let's see about wrapping this up.

Good night.

Coding 2025-05-08

2025-05-08T04:00:00-04:00

Well, I got some stuff done today, and it's kind of too late for a detailed writeup, so let's see what I can manage without details.

So, I've been focused on supporting user-defined taxonomies that may contain metadata fields. I'm right now assuming that it's reasonable to require that parent relationships in the source file should point to earlier in the file, because that makes the implementation very simple (relatively speaking) and doesn't seem like an undue burden from an authoring perspective. (And I can add the requirement to canonicalization, too.)

One thought I had writing some of these methods is that I'd like to bring in deal as well. Maybe tomorrow.

Anyway, the big problem right now is that this is kind of disorganized, and I am, on reflection, not sure if I want to keep going with some of the "neat ideas" I had. I think the way forward at this point is to document what I'm trying to do, and to figure out what's missing relative to that plan.

For now, I need to get ready for bed and wind down.

Good night.

Coding 2025-05-07

2025-05-07T04:00:00-04:00

The ideas I was trying to work out are developing, and things look generally promising. There are a lot of TODOs remaining, some of which I'm just keeping track of in my head, so that's... not ideal.

I'll try to write up the state of things tomorrow, because I ended up staying up way too late.

Good night.

Weekly Roundup 2025-05-06

2025-05-06T04:00:00-04:00

Wednesday: I tried to better document the relationship between the conlang grammar source file and the runtime representation.
Thursday: Migraine.
Friday: I made some progress on the code for conlanging stuff, but I also made some mistakes that kind of embarrassed me.
Saturday: I was tired.
Sunday: I was able to think about the conlang stuff again.
Monday: I got most of the way towards figuring out what I want the source representation to look like, but I'm still not there when it comes to the runtime representation.

Next week, I'm going to try to update the runtime representations and write more structuring logic, but I can't promise it'll get anywhere anytime soon.

Coding 2025-05-05

2025-05-05T04:00:00-04:00

I've got a good idea of how I want to represent the different kinds of word taxonomy in the source file for this conlang grammar stuff, but I'm not quite there when it comes to the runtime representation.

It's like, I'm dividing words into two kinds of taxa, and currently I believe one kind of taxa wouldn't have any metadata associated, but the other kind would, but also validating (or, worse, generating) that metadata requires reference to other parts of the taxonomy. And this has to happen in, like, several orders. And the root allows metadata that all other nodes do not. I'm not totally sure how to force this all to work, and the weird relationships have me thinking that I probably shouldn't try to force it together; I need some inspiration...

Anyway, it's late and I need to get ready for bed.

Good night.

Coding 2025-05-04

2025-05-04T04:00:00-04:00

I was tired, so I just had the focus to ponder some of the conlanging software design, and I'm worried that I'm taking things in an excessively complicated direction, but then again, languages often get complicated.

Basically, there are different kinds of taxonomy that words can fit into. We can consider the ending of verbs in Spanish, for example, or the animacy of nouns in Spanish. The former case influences the realization of different conjugations, while the latter case influences inflection for gender. Back to verbs, stuff like tense can affect what a verb conjugates for.

What I'm trying to figure out is a non-onerous means of representing this stuff in the source file. I'll see if something occurs to me soon, but if not, I'll have to switch gears.

For now, though, I'm going to wind down.

Good night.

Diary 2025-05-03

2025-05-03T04:00:00-04:00

I'm doing better today, but I was really tired. I did have some thoughts about how to improve my code, but mostly I ended up trying to take things easy.

And I'm going to continue with that, rather than trying to come up with something to say.

Good night.

Coding 2025-05-02

2025-05-02T04:00:00-04:00

I spent a bunch of time customizing the structuring logic for the conlang toolkit stuff. I've made some progress, but ultimately, I think I may have pushed myself too hard. If I were in better shape, I think I would have noticed sooner that I was calling a method on the wrong cattrs converter; this sent me down a rabbit hole of wondering how it was that no method was having any effect. But at least now I've got the stuff that was broken there fixed forever.

Anyway, I definitely shouldn't go over anything else right now, or stare at my screen any longer.

Good night.

Diary 2025-05-01

2025-05-01T04:00:00-04:00

Two nights ago, I realized some things about how I'm using my time.

Last night, I committed to focusing on a broader range of hobbies.

Tonight, I'm fighting off a migraine and I just want to curl into a ball.

Fingers crossed I'll get something done tomorrow?

Good night.

Coding 2025-04-30

2025-04-30T04:00:00-04:00

I'm messing around with a bunch of things today, and planning to have things ramped up a bit, but I do have a little to write about quickly.

I went over the TOML and attrs representations of the different parts of the grammar, and noted them next to each other. This is necessary for figuring out what the structuring code is going to have to look like, since I'm trying to apply principles of ease-of-use to both representations, which, given the different underlying formats, means that they don't always match up very closely. For now, I just focused on documenting the format; next, I can work on making sure that each class structures properly, starting from the leaves and working inward.

The other thing I'll work on, once structuring appears okay (and I'm therefore confident that I've got a handle on the different representations), is the canonicalization process. That'll be... something.

Anyway, I should wrap things up now.

Good night.

Weekly Roundup 2025-04-29

2025-04-29T04:00:00-04:00

Wednesday: I got a bit of work done on MOTR.
Thursday: I got a bit more work done, and it revealed some fascinating bits of breakage.
Friday: I did some small polish work on MOTR, then put it down for a bit.
Saturday: I messed around with some code that, unlike MOTR, which I wish were already generally usable, is entirely a solution in search of a problem.
Sunday: I switched gears to the grammar pipeline for my conlangs, and staked out some basic questions to answer.
Monday: I worked on those questions, and ended up raising more, more complicated questions, and we'll see where this all ends up.

Next week, I'm going to try get a handle on things in the grammar pipeline, and also I've been thinking more about music lately, so I'm thinking I'll try to do something with that.

Coding 2025-04-28

2025-04-28T04:00:00-04:00

Okay, here we go.

First off, my prototype for the grammar source does appear to be syntactically valid TOML. Good for me. Secondly, the layout of what's there does look about right. As to whether this all works, that remains to be seen, because I'm still working on some choices about the right way to model this stuff with custom classes.

The big question that I have is whether it suffices to make the runtime representation closer to the output format (in particular, whether I "need" to have different classes for tables that have different representations in TOML), or whether I want the representation to be close enough to the file that it's possible to round-trip between tomlkit classes and my own classes.

And whether I care about round-tripping like that depends on whether the canonicalization logic I have in mind for the configuration file can be written purely in terms of the tomlkit structures, or whether that represents a duplication of effort. And I suppose that depends on whether I assume that I'm always unstructuring canonicalized data, or if it makes more sense to incorporate the canonicalization into the structuring process. At the same time, I only need to worry about un*structuring logic in the case that I *am canonicalizing via round-trip. Either way, talking through this has gotten me to figure out a slightly better representation for some of this, so I'm going to go implement that quickly.

...

Okay, back to "where do we canonicalize?". Thinking about this a little further, I don't want to have to be responsible for tomlkit classes inside hypothetical unstructuring code, so let's see if pre-cattrs canonicalization is feasible. Canonicalization is partly a question of handling a default value for one content type, and getting the sorting of various parts of the file correct:

The sections list should match the ordering of the table of contents
The lexicon should be alphabetized by stem
A variety of things should have field orders matching their runtime definitions, except that whenever the spec field appears, it should be first
The cells of sparse tables should be sorted with the rows as the most significant radix, and the columns as the least. (Technically, I guess this means that canonicalization has to involve some level of structuring, even if it's not returned anywhere, because sometimes the canonical order is specified elsewhere in the document.)

Because that third bullet point seems pretty obnoxious to handle in the context of generated unstructuring hooks, let's assume that we're going with pre-unstructuring canonicalization.

Actually, now that I look at this with slightly fresher eyes, I see that I need to reconsider one of my table definitions, and either change one that currently exists, or add a new one.

...

I think I came to a reasonable decision, which I can always revisit as I tweak this stuff further.

For now, I want to wind down for a bit.

(Actually, I was thinking a bit more, and now I'm not totally sure how I want to handle some of these tables, so that's going to be, interesting.)

Good night.

Coding 2025-04-27

2025-04-27T04:00:00-04:00

I had kind of a rough week, so I ended up trying and failing to take things easy (hopefully I'll have an easier time of it tomorrow), but I did get some work done on prototyping the grammar for my Untitled Personal Conlang project.

I think what I should focus on next is taking the current state of the prototyped source file, and check for things like:

Is this actually syntactically valid?
WHen I deserialize it, does it have the layout that I expect?
Can I actually interact with the data in the way that I expect to be able to?

Those are all good questions, and they should be possible to answer without any undue effort. I'm not going to try right now, though, because it's late.

Good night.

Coding 2025-04-26

2025-04-26T04:00:00-04:00

Well, I was out of it today, but I did make some improvements to the stuff I was messing with yesterday.

A few weeks ago, I started implementing an int subclass in Python that presents itself as balanced ternary instead of two's complement, using Kleene logic for the trit-wise operations. Yesterday, I started implementing format spec extensions for it, to print it as trits instead of decimal digits or anything else. Today, I cleaned up the way I parsed out format spec information to make things a little cleaner and simpler.

I might come back to this more, but probably not right now. What I'll work on next... I don't know, we'll see.

Good night.

Coding 2025-04-25

2025-04-25T04:00:00-04:00

I did the very obvious cleanup on MOTR, but I'm not going to push myself any further because the sun was bright and I had a rough afternoon as a result.

Actually, I did just remember one other thing I was thinking about that could be a little amusing. Let me see if I can do anything with that idea. I will not be elaborating or reporting back.

Good night.

Coding 2025-04-24

2025-04-24T04:00:00-04:00

I spent a good chunk of time getting my planned changes working, and in the end, I made a bunch of compromises because I want to do anything else with my time right now.

Expanded the types of a helper function so much that they crashed my plugin. The correct course of action would be to create separate helper functions each specialized to the correct degree. This would also allow me to have better error reporting, out of these functions that only exist to report errors.
Some of the new Axes methods have confusing names; I should try to address this.
In the end, I concluded that some aspects of my reference implementation are "wrong", and I slightly redid the filename generation in one wrapper under the assumption that the current behavior is "right". This raises two questions: can I explain the current behavior, and does anything else need to be changed?
Maybe there are other corners I cut. I'm tired, okay?

I'm going to wrap this up now so I'm not tempted to poke at it any more than I already did while writing those points up.

Good night.

Coding 2025-04-23

2025-04-23T04:00:00-04:00

I have carried out the first part of the plan from two days ago (lightly modified), and it sure seems to be working okay. I had to rewrite a few tests for various reasons, and I think I'm missing coverage on the new validators I wrote.

Regardless, this is a very good stopping point for now, and I'll see about making the rest of the changes tomorrow.

Good night.

Weekly Roundup 2025-04-22

2025-04-22T04:00:00-04:00

Wednesday: I was making progress on the conlang stuff, and then I remembered that MOTR isn't generally usable yet, grr...
Thursday: I did some work on MOTR, and also managed some conlang work still.
Friday: I discovered that MOTR had some significant testing misses and stuff somewhere between "missing feature" and "serious bug".
Saturday: I started planning out how to get test coverage of this stuff, so it hopefully won't randomly regress.
Sunday: I got some tests written, and pondered how to get them to pass.
Monday: I did a bunch of planning for how to get the tests to pass, and I have... already decided to change some of those plans.

Next week, I'm going to try to carry out those plans, and we'll see if I change my mind further, or if anything goes wrong.

Coding 2025-04-21

2025-04-21T04:00:00-04:00

As I alluded to in the summary, I'm intending to rework some of the Axes fields. The field in question is called box_labels, and the plan is to supplement it with two additional fields. These supplementary fields would be called something like defaultable_labels and defaulted_labels, which satisfy box_labels >= defaultable_labels >= defaulted_labels. One of the key derived fields would have a value of box_labels - defaulted_labels.

I'm currently looking over the usages of box_labels to see what would need to change.

defaultable_labels and selection_labels must have an empty intersection.
set_defaults_for_labels must add any defaulted values in defaultable_labels to defaulted_labels.
product has to do a bunch of extra calculations that I've previously worked out.
selections must check in box_labels - defaulted_labels, instead of just box_labels.
draws_from_box must set defaultable_labels as well as box_labels.
uses_own_defaults requires a new Axes method that I have previously worked out the behavior of.

Once that's in and I've confirmed that everything still works, my next step is to convert the inputs field from a Box to a DependentMapping[Callable[[Label[T]], tuple[T, Box]]], and make the following updates:

Wait hold on, I'm not sure if this fixes the problem I encountered.
Hm.

Basically, I want to know the default values that are relevant for the calculation of a Matrix associated with an Input. Now, the Matrix that goes into a PreInput will carry some of that information, and it shouldn't be possible to get it wrong, per se. What's missing is the defaults that got set after the PreInput was generated.

Okay, this should work, actually.
Basically, when calling Axes.set_defaults_for_labels, it needs to now pass the entire Box, not just the set of labels, and that Box has to be added to the right of the Box values in inputs. Question: can/should the values of default_arguments be filtered based on the contents of defaultable_labels?
with_input must process its argument accordingly.
without_inputs must use a different type value in order to typecheck.
product has to... do... something... probably it should actually just keep doing what it's doing? Very confusing. Maybe it should be taking either the union or the intersection? Maybe needs to be checking against the other Axes fields. On further consideration, it should simply keep the same logic.

Well, that was a bit of a ride. I don't want to prolong it for tonight, so I'm going to wrap up for now, and see if this all still makes sense when I come back to it.

Good night.

Coding 2025-04-20

2025-04-20T04:00:00-04:00

Okay, so I got some tests written that are revealing one of the bugs I want to fix, and I just wrote another test that covers the bug I want to fix first. I've sketched out some ideas for how to address that bug, after which I should be able to write the functionality needed to address the second bug, which I noticed first. I'm not ready to try to put those sketches into practice, but they'll probably work.

I'd better wrap up, and try this out in a few days.

Good night.

Coding 2025-04-19

2025-04-19T04:00:00-04:00

Okay, let's see... I was pretty tired today, so I didn't get as much done as I'd like. Let's see if I can sketch out the next steps I want to take, to make sure I can course-correct with MOTR.

I have a set of tasks. Some of these tasks are Cmd objects. The most important thing to look at about them is their lists of arguments. What I want to do is match on the contents of the argument lists. Two kinds of match are relevant: one is to treat the string as a path, and only look at the final segment; the other is to do an exact string match. My hope is that it will suffice to match one or two arguments. Once I've isolated a single Cmd object to inspect, I want to write assertions around their contents. Some of these assertions should be of the form "after this argument, expect this other argument". Others might need to be purely positional, which seems a little dicey. I'll need to think on this more, and get to bed for now.

Good night.

Coding 2025-04-18

2025-04-18T04:00:00-04:00

Things have gone wrong with the attempts to make MOTR work better. Well, wrong in product code, right in test code, because exactly one test managed to exercise a large amount of functionality, which was the bare minimum to demonstrate that my current ideas for Matrix chaining simply do not work.

In my head, I'm rapidly flipping between different solutions, and none of them yet strike me as especially good.

Basically, what's happening is that some code has the ability to set Box values, provided that nothing upstream sets the same key, and the chaining implementation causes various things to suddenly be upstream from other things, when they weren't before.

The desired behavior, I think, is that any chained Matrix will only see the initial Box value, and not any values that got added "in the interim".

Currently leaning towards, track the "initial" Box and the "default" Box through the call stack, and only combine them for draws_from_box.

Okay, I've thought about this enough that I've got something to try now. As I type this, it is running...

And it passed. Okay, next hurdle.

Well, hurdles.

The new interface I just switched to causes one of the new names to not really make sense.
I think some of the checks I'm doing are redundant now.
I may need to perform additional checks now.

Those last two do not cancel out. Or maybe they do. I'm remembering that this code doesn't all fit in my head at the same time. Oh boy, it looks like all of this code just barely does something sensible. I'm going to have to check this over later.

Unfortunately, I didn't get time to get more done conlanging, but what I did get done was certainly an adventure.

Dangit, I just checked, and I'm missing some key functionality in what I've been writing for years. I'm going to need to come up with some better tests. Working on this is also going to need to wait for now. It's late.

Good night.

Coding 2025-04-17

2025-04-17T04:00:00-04:00

I took things relatively easy today, and I'm going to try to unwind after I publish this, but here's what I got done today:

Rewrote the matrix module so that the helpers are in simple present, rather than imperative or a secret third thing, and clarified or improved some terminology choices.
Added more of a skeleton to the grammar for Untitled Personal Conlang 2025.

Here are the next steps on each side of things:

Remember exactly how I intend to use the new prevents_multiple_returns_from helper to sequence between Matrix objects.
Fill in the stuff for UPC2025 that I know about.

I'll look into that over the next few days, but right now I just want to find something else to unwind with.

Good night.

Coding 2025-04-16

2025-04-16T04:00:00-04:00

I've been sketching out format details for the grammar, and it'll probably work. I just need to take a bunch of notes to translate into the source file so I can validate the code against the kind of stuff I want to write.

However, before I write that code, I need to consider, do I want to write tests for it, or just resign myself to debugging, and if the former, I guess I've got to bang out a tox.ini file really quick, ugh. It's annoying but true that MOTR isn't ready for general usage yet, and can't get there quickly. I think I'm going to also be trying my hardest to make sense of the matrix module, and How To Do Better.

Anyway, none of that's happening right now; I need to get to bed.

Good night.

Weekly Roundup 2025-04-15

2025-04-15T04:00:00-04:00

Wednesday: I got fast error diffusion written.
Thursday: I cribbed fast-enough code for generating blue noise.
Friday: I developed a fancy algorithm for generating space-filling curves, and I literally cannot tell if it was worth it.
Saturday: I squinted at too many washed-out images that some algorithm or other seemed to indicate was "the good version".
Sunday: I switched gears back to conlanging, and decided to scale back some of my ambitions.
Monday: I set up some infrastructure for this and future projects.

Next week, I'll start properly planning and prototyping the grammar generation code. Or something. I don't know.

Coding 2025-04-14

2025-04-14T04:00:00-04:00

Okay, I've been tweaking my tooling setup, and it looks like things should be good to go soon.

I've got one repository for the tools to process the grammars.
I've got one repository for working with the specific grammar I want to write.
I've got a pypiserver set up and running on login, to let me take a package published from the first repository, and use it in the second.

The first repository is mostly set up as far as tooling goes, and I need to take a little time to draft things out in the second repository, so I can figure out what kind of interface to give the first repository, and then finally write build rules for the second. Nothing's happening with that tonight, because it's late now and I'm tired.

Good night.

Coding 2025-04-13

2025-04-13T04:00:00-04:00

I ended up stepping away from the thermal printer stuff entirely, and focusing on a personal conlang project. For this project, I'm deliberately stopping myself from doing some things "the right way" according to some people. I'm instead trying to put things together in a way that's tractable in my head, and then I can potentially scale things up as I manage to automate more and more of the essential process.

Here's my general plan:

Data about the conlang is typed up in TOML files.
These files are canonicalized using tomlkit.
In addition, they are deserialized using tomlkit, then fed into cattrs to produce standardized data in frozen attrs classes.
This data is in the form of a hierarchical tree structure, which feeds into a Jinja template, to produce an HTML document.
This HTML document is converted into a vaguely academic-looking PDF using pandoc.

I know this looks like a lot, but almost all of this is stuff I've done before, so it should be relatively easy to snap together, and if I need to change some aspect of the flow, to snap it back apart.

The big stuff I'm trying to work out currently is how I want to build things. Tup sounds cool, but I've never quite managed to convince myself that the stuff I want to use it for is the stuff it's actually meant to do, so I figure I'll crib from the Ninja configuration stuff I've written for other projects.

That's about enough to think about for tonight.

Good night.

Coding 2025-04-12

2025-04-12T04:00:00-04:00

The more I ponder and compare different space-filling curves, the less sure I am that working on them is worth anything outside of the sake of working on them. Like, they render cool pictures, but every attempt I've made to evaluate objective functions for image processing quality has managed to find at least one image where my subjective opinions about what look good have nothing to do with the parameter values that maximize the objective function, so, like, why would I trust these functions to say anything about the relative quality of different Riemersma dithering implementations?

With all of that said, I need to really step away from this stuff for a while, and focus on generating images from HTML and SVG or whatever.

I don't have anything else I want to talk about right now, so I'm going to wrap things up.

Good night.

Coding 2025-04-11

2025-04-11T04:00:00-04:00

I have implemented the space-filling curve Ideas, with only one show-stopping bug. I'm not sure if the resulting curve, the Glittering Curve, is any good in general, but I'll have to put off working that out to some other time. For now, sleep.

Good night.

Coding 2025-04-10

2025-04-10T04:00:00-04:00

I swiped some public domain code for generating blue noise, and the result looks solid. I would like to factor the code apart a little, but right now it can generate a 64 by 64 texture in like a second, so, like, should I actually care?

At this point, I can either start messing around with alternative space-filling curves or averaging algorithms for Riemersma, or start playing around with image generation, now that I've got a good stable of algorithms. Realistically, I should be focusing on what I'm going to actually print, but I have Ideas about space-filling curves...

Anyway, I should get to bed twenty minutes ago.

Good night.

Coding 2025-04-09

2025-04-09T04:00:00-04:00

Summary of the results of rewriting the dithering:

Trying to do Fancy Numba Stuff in an Idiomatic Python Way didn't really work out. I still managed to do Fancy Numba Stuff, but I don't like how it looks.
I had some bad code in the slow version, which Numba caught, and I think fixing that code actually made the slow version a little faster.
I haven't tested this too much, but compiling then running the jitted code is faster than the faster slow version, and once it's compiled, I saw a speed up in one case of I think like a factor of 15.

So, all in all, this is working out well so far. Unless I'm forgetting something, I guess next I look into blue noise generation. (I guess I should confirm that my rewrites didn't break Bayer dithering first.)

...

Okay, yeah, Bayer dithering is working fine, and when I retested error diffusion, I saw some even more dramatic speedups. So I'm really enjoying the new toy here. Regardless, I need to get to bed right now. I'll try to polish things up a little tomorrow, and then to figure out blue noise.

Good night.

Weekly Roundup 2025-04-08

2025-04-08T04:00:00-04:00

Wednesday: I got matrix-based error diffusion working, and started experimenting with it.
Thursday: Taxes.
Friday: Couldn't do coding work because eyes hurt.
Saturday: Couldn't do coding work because I dunno.
Sunday: I listed a bunch of the stuff that I wanted to work on, but it wasn't working out.
Monday: I drafted out all of the new versions of the code I'm interested in having in the near future.

Next week, I'm going to try to update the thermal printer code, but I'm not going to start on that for now, because I just got sidetracked by a problem I made for myself today.

Coding 2025-04-07

2025-04-07T04:00:00-04:00

All right, I didn't want to touch the code, but I've written up all of the changes I want to make to the code, and all of the ideas I have for parts of the code that I don't need yet. As part of all of that, I typed up rough drafts of the jitted versions of the error diffusion dithering algorithms. (Well, except for Riemersma. I want to put Riemersma on the back burner for now, because everything else just works so much better, currently.)

Because I don't want to update Riemersma, or delete it without documentation, my plan right now is:

Move the dither module to dither_old or something.
Create a resize module to make things be the right size.
Create a colorspace module to handle getting images down to linear luma.
Create new dither module to host the more ergonomic, hopefully-faster functions I've drafted.

With all of those together, I should be able to make the testing I've done clearer, and faster. We'll see, though.

Good night.

Diary 2025-04-06

2025-04-06T04:00:00-04:00

I did some important things and had some important conversations today, but as far as stuff that I'd consider for a blog writeup goes, I'm kind of spinning my wheels. As far as the thermal printer goes, I haven't figured out how to document the image processing code that I'm currently not planning to implement. As far as MOTR goes, I haven't found a good place or person to discuss what needs to change about the matrix module. I haven't had the focus to work on Clifford chemistry stuff. I've got a game concept that I don't think has quite reached half-baked. I've got various other pieces of writing that I don't intend to discuss here.

For now, I'm going to try to wind down and see if I have the focus to work on any of this tomorrow.

Good night.

Diary 2025-04-05

2025-04-05T04:00:00-04:00

I still had not much focus today. I'm going to wrap up now and try to get some sleep.

Good night.

Diary 2025-04-04

2025-04-04T04:00:00-04:00

I should be done with all of the big stuff that I need to take care of this week, so I thought I'd be ready to start updating the image processing code for the thermal printer. But like, man... my eyes hurt. I'm going to go do one or two things, and then just do some nice frontlit reading.

Good night.

Diary 2025-04-03

2025-04-03T04:00:00-04:00

Uuuugh. I wanted to work on improving the interfaces for dithering images, and instead I spent all night filing my taxes. At least it's done.

And I have stuff to do at work tomorrow that I don't want to...

I'm going to publish this and go read.

Good night.

Coding 2025-04-02

2025-04-02T04:00:00-04:00

I bit the bullet and implemented matrix-based error diffusion. Notes so far:

When I do Floyd-Steinberg diffusion, I don't get what Pillow does when it does Floyd-Steinberg diffusion.
Sometimes what Pillow does is better in terms of detail, and sometimes it isn't.
Sometime Floyd-Steinberg is good, sometimes Jarvis-Judice-Ninke is good, and I don't think I've yet hit a situation where Atkinson was the standout. I probably need to try a greater variety of images.
At least the way I write it, matrix-based error diffusion is incredibly slow.

Looks like I should work on a few categories of thing later:

Cleaning this up and reworking the relevant interfaces.
Finally generating blue noise in the format that I want.
Some quick experiments showed that simply increasing the input image brightness (by quite a lot, in the case of Atkinson dithering) can make things come out a lot clearer. Perhaps there's some way to automate the offset determination?
I've still got a few more average implementations I want to try, but, like, realistically speaking, they're not likely to accomplish anything groundbreaking. Nothing with the aesthetic punch of "Atkinson dithering but first I added one eighth of full brightness are you kidding me?" (I have no idea what this corresponds to mathematically, or how much it messes with the theoretical properties of Atkinson dithering.)

Anyway, we'll see if I'm up for any of that tomorrow.

Good night.

Weekly Roundup 2025-04-01

2025-04-01T04:00:00-04:00

Wednesday: The dithering code I wrote was super broken.
Thursday: I decided to move away from Python REPL for prototyping the dithering code, and made a plan for what changes to make at the same time as switching to a notebook.
Friday: I made some progress on that front.
Saturday: I wrote a bunch of helper functions that I'll eventually want to incorporate into the project proper.
Sunday: I fixed a bunch of the aforementioned super-brokenness.
Monday: I looked over what I had, and took notes and made more plans.

Next week, I'm going to try to finally get matrix-based error diffusion going, and perhaps blue noise generation.

Coding 2025-03-31

2025-03-31T04:00:00-04:00

I kind of took things easy today, so let's instead take some notes.

Bayer level 3 seems like an extremely good default choice, especially if I mess with the luma or contrast by a few percent.
Riemersma doesn't seem like much more than a party trick currently; I'll have to implement different error decay generators and see how those do.
I'm still not sure how I want to encode error diffusion matrices.
I'm sort of leaning towards PMaps from coordinates to floats. (With validation that the y coordinate is non-negative, and if the y coordinate is zero, then the x coordinate is positive)
Searching for example code gets me stuff that inlines the logic of selecting different coordinates. This makes sense from an efficiency perspective, probably, but I'm trying to get an idea of how a bunch of different algorithms treat images.
I'm also seeing something that combines scaling and resampling into a single step, which, I mean, that might help, but I'm not up to thinking about it currently.
Pillow's Floyd-Steinberg implementation seems to make everything much lighter? Not sure if that's an artifact of the algorithm, or if there's something funky in the implementation.
I guess I should also be looking into blue noise stuff, if Bayer seems so reliable, but I don't feel like it currently.

I'm going to ponder this for a bit, and wind down. Hopefully I'll get some kind of implementation scraped together in the days to come.

Good night.

Coding 2025-03-30

2025-03-30T04:00:00-04:00

All right, I got some testing done in a notebook, and figured out what was wrong with my Bayer dithering implementation; now that I've fixed it, it can be excellent for some images.

Riemersma dithering was fun to implement, but so far I haven't found an image where I feel that it really shines. There's some where it does the best job out of what I have easy access to so far, but I'm guessing that could change once I implement Atkinson dithering.

Pillow, as I might have mentioned before, provides Floyd-Steinberg, and sometimes this is fine, and sometimes it kind of isn't. Hence why I'm interested in other forms of error diffusion.

Anyway, I ought to wind down now. Hopefully tomorrow, I'll have time to wrangle matrix-based error diffusion out of Numpy's API.

Good night.

Coding 2025-03-29

2025-03-29T04:00:00-04:00

I started working on a notebook for testing this stuff out. I got a few helper functions written:

Resize images to fit on the thermal paper
Linearize a channel
De-linearize a channel
Given an image, convert it to a linear grayscale image

This all looks like a proper baseline to be testing this stuff out, and I'll try to get on that tomorrow. For now, I'm going to take things easy.

Good night.

Coding 2025-03-28

2025-03-28T04:00:00-04:00

All right, I've factored out the averaging algorithm. Now, I feel ready to start prototyping stuff in a notebook, but I probably shouldn't mess with that tonight.

Instead, I should wrap things up and get ready for bed.

Good night.

Coding 2025-03-27

2025-03-27T04:00:00-04:00

Whoops, I figured out what was wrong with the dithering stuff I was testing. I was attempting to set up an exponential moving average, and I wasn't properly setting it up. Since figuring that out, I've found out or worked out a few other things:

The current implementation seems to work okay for photos, and a high alpha value makes text look interestingly crunchy, while low alpha values make text look smooth, and can unlock The Forbidden Edge Detection on photos.
On reflection, it should be possible to abstract the averaging algorithm out of the dithering function, and pass it in. Possibly also the space-filling curve generator. (While I'm at it, it'd be nice to be able to tweak the threshold in the space-filling curve generator, or even change the algorithm.)
I've been testing the prototypes just in Python's REPL, and I think I should switch over to using notebooks.
I've downloaded a bunch of stock photos to experiment with, instead of the not-stock images I was messing with before.

My basic plan at this point is:

Abstract out the averaging algorithm.
Make a notebook for testing out different algorithms and parameters, and prototyping helper functions that I wish I'd had like a week ago.
Figure out what I'm doing wrong when it comes to Bayer dithering.
Implement more algorithms: different forms of moving average, different space-filling-curve thresholds, different space-filling-curve generators, matrix-based ordered diffusion, blue noise generation finally.
Somewhere in all of this, make some kind of pipeline object.

For now, I'm going to think about how to factor out the averaging algorithm, because I tend not to do much with the send() method in Python.

Good night.

Coding 2025-03-26

2025-03-26T04:00:00-04:00

I thought I was going to be able to throw together dithering pipelines easily, and I guess I can, they just don't really work.

The Bayer dithering is artifacting heavily, in ways I didn't expect, and the Riemersma dithering (which I could be doing just completely wrong) is hitting exponential blowup.

I would like to be able to figure out what I did wrong there, but I don't have the time to ponder it tonight.

Good night.

Weekly Roundup 2025-03-25

2025-03-25T04:00:00-04:00

Wednesday: I ran into some trouble with the thermal printer. I don't recall that I fixed it to my satisfaction.
Thursday: I did manage to get it to work for other cases, at least.
Friday: I did further experimentation with the printer.
Saturday: I experimented with the dithering options in Pillow, which are pretty limited, so I'm planning to roll my own.
Sunday: I made some progress towards rolling my own versions of various kinds of dithering, but also the project repository is a mess.
Monday: I cleaned things up, and potentially got some forms of dithering working, though I haven't actually checked.

Next week, I'll try to make more progress, and maybe write up the space-filling curves stuff, since I did have to put some effort into understanding what's going on, and also it's possible that I can get rid of that corner case without too much effort.

Coding 2025-03-24

2025-03-24T04:00:00-04:00

My Jujutsu state is, if not clean, capable of basically pretending, so that's nice.

After I got that together, I wrote the framework code for applying different dithering algorithms, and what should be enough code to pull off Bayer dithering, though I haven't tried it yet. I also added some convenience methods to the class that generates images for the thermal printer, to do stuff like load images after they've been through the dithering pipeline, and preview images from whatever source.

On the fancy-code-side of things, I still haven't wired up the space-filling curve code to handle Riemersma dithering, and it would be nice to special-case things a little to avoid the corner case (which manifests in a corner) in the current space-filling curve code. I also haven't set up blue noise generation, or matrix-based error diffusion.

On the usability side of things, I haven't tried actually wiring any of this stuff together, so my standing assumption is that actually trying to use this code is really obnoxious currently. I probably need a bunch more helper functions that I haven't fully anticipated.

I'll need to take some time to figure out how to implement things, and what things to implement, so I'm just going to wrap things up a little early and chill out for a bit.

Good night.

Coding 2025-03-23

2025-03-23T04:00:00-04:00

Okay, I've got prototype code for generating "nice-enough" space-filling curves, modulo a minor cheat in what is technically a corner case. This covers part of what I need for Riemersma dithering; the rest is handling the actual quantization etc.

In terms of neat code stuff, the next things to do are to set up the overall pipeline logic and write a debug function that handles the fact that light is dark and dark is light.

In terms of project workflow stuff, I really need to get back to working with Jujutsu, and figuring out how to make some kind of reasonable commit out of the complete nonsense I've managed to accrue as a result of not wanting to think about how Jujutsu works.

We'll see if I have the discipline to do that, tomorrow. For now, bed.

Good night.

Coding 2025-03-22

2025-03-22T04:00:00-04:00

I spent a lot of today just messing around with the dithering options that some tools give me out-of-the-box. These options are pretty limited, so I also spent some time sketching out implementation strategies for more advanced options. I think some of them are ready to be worked on, but others are... um... well... the cyclomatic complexity is going to be pretty big probably.

Anyway, I really was playing around with dithering images a lot, so it's quite late and I should get ready for bed half an hour ago.

Good night.

Coding 2025-03-21

2025-03-21T04:00:00-04:00

I made some tweaks to my helper function for printing, which allowed me to confirm that sending images in two separate print commands will directly concatenate them vertically. This is good, since it means I can more-or-less arbitrarily split images up if they're too big.

Aside from that, I mostly focused on planning out image processing. This planning is a mix of figuring out what I want stuff to look like, and what Numpy will easily support.

In any case, I have some ideas, but not time right now to act on them. Got to get ready for bed.

Good night.

Coding 2025-03-20

2025-03-20T04:00:00-04:00

I decided to focus on printing single images from the thermal printer. This should give me enough control, and means I don't have to puzzle through the parts of the XML that don't feel like XML.

My plan to build on this is to implement different dithering algorithms, apply them to some test images, and make sure they come out all right. After that, I'll focus on generating images by using a webdriver to read HTML, and then develop templates.

But for now, I'm pretty tired, and just glad that I managed to make it output a literal black triangle.

Good night.

Coding 2025-03-19

2025-03-19T04:00:00-04:00

I decided to focus on generating and printing images, because that seems more versatile. So far, it hasn't really worked, so I'll be troubleshooting that tomorrow, because I spent all night throwing together this manual test that didn't work.

For now, I should get ready for bed.

Good night.

Weekly Roundup 2025-03-18

2025-03-18T04:00:00-04:00

Wednesday: I had some uncertain thoughts about Clifford chemistry.
Thursday: I tried to elaborate on those thoughts, and came to the conclusion that I need some kind of four-dimensional molecular modeling software, and I have no idea where to look for such a thing.
Friday: I eventually managed to plug the thermal printer in. This was harder than one might reasonably expect.
Saturday: Were the WSDL files ever real in the first place? I don't know, but there is extensive documentation that tells me most of what I need to know.
Sunday: I managed to construct an empty print job programmatically. This is actually useful.
Monday: I actually started to understand the documentation for the thermal printer.

Next week, I'm going to try and implement enough bit-twiddling stuff to print images out, and try to generate them.

Coding 2025-03-17

2025-03-17T04:00:00-04:00

I mostly took things easy today, but I did do a small amount of experimentation with the printer. I've now got a general idea of how I should expect things to fit together. Basically, there are a few natural groups of XML nodes, and they mostly do not nest, so I should rework things a little. I've bookmarked the relevant pages in the XML manual, so I should be able to throw something together, once I've re-evaluated the constraints on the layout.

And, let's see... some nodes are exclusive to page mode, some are exclusive to standard mode, some are shared, so I believe I want to have a common base class, and mode-specific subclasses, because some attributes are also mode-specific.

The big thing I need to ponder right now is how to handle stuff like <vline-begin> and <vline-end>, which are meant to be paired, but don't have to go in a particular order. This is something where I don't see an obvious way to handle it with closures or context managers. Like, I guess it's a linear logic sort of thing, but I'm not sure how to get that into any kind of easy Python paradigm.

Anyway, I think the right way to go about this is to make myself a checklist of all of the elements and their modes, and figure out what kinds of validators and enums I want to write for them.

Anyway, I need to get to bed right now.

Good night.

Coding 2025-03-16

2025-03-16T04:00:00-04:00

Well, I did end up developing a different idea for how to handle the tree, of saying "well, let's build up the tree structure without the node data making any reference to it". There are probably a few rough edges that I'll hit later on, but I've managed to get it to output the XML for a blank document, which should be enough for everything else to fall into place from the documentation.

I'll see about making it do something actually interesting later, but for now I would like to wind down so I hopefully don't end up sleeping in quite so much tomorrow morning.

Good night.

Coding 2025-03-15

2025-03-15T04:00:00-04:00

All right, here we go. Slow progress, but progress.

First off, I'm sure I read so much about the WSDL files, but I can't seem to find anything no matter how much stuff I download. (Also, why is the JavaScript SDK only available under the Windows downloads? That's weird, right?) So, it looks like I need to build up the XML myself.

What I've got right now is a bunch of work that only exists in my Python REPL's readline history, as well as some XML payloads that work and some that don't. Once I collate all of this, and bring in the documentation, I should have what I need in terms of controlling the printer.

Before I touch that, though, I need to think through what kind of interface I want to use to build this stuff. From my experiments so far, I believe "classes representing the different node types, defining functions that mutate an instance of the parent node" should offer a good balance of features, ease of implementation, and maintainability.

I'll sleep on this and see if I come to any different conclusions.

Good night.

Coding 2025-03-14

2025-03-14T04:00:00-04:00

All right, after going through some interesting stuff with shipping (why did they put "receiving department" on the label if they knew it was a residential address? Baffling) I've got the thermal printer, and it definitely seems to be working.

Now that I've taken care of the hardware side of things, I've got to work on writing support code of varying levels of sophistication. Here's how I believe things go:

Use the firmware version information to finally access the relevant WSDL file
See how interfaces generated from that file feel/work
Experiment with images, first pure black and white, then greyscale subject to various forms of dithering, then start working on generating images

This will have to wait until the weekend before things have a chance to get interesting. I have a job, after all. And for now I should be getting to sleep.

Good night.

Diary 2025-03-13

2025-03-13T04:00:00-04:00

Well, I thought some more about Clifford chemistry stuff, but it amounted to "*half a page of hopefully post-rigorous math* So therefore this doesn't work ... Oh wait, haha, one of my assumptions is wrong, so I don't know if my conclusion is valid."

If I'm going to get anywhere interesting modeling this stuff, I think I'm going to need some sort of formal system, hopefully something I can check, and maybe even construct, electronically.

For now, I really really really need to get to bed, so maybe I'll look into that tomorrow.

Good night.

Diary 2025-03-12

2025-03-12T04:00:00-04:00

Oh boy. I did not have the focus to do much today, at least not that I feel like writing down. I figured out some of the potentially-biologically-significant aspects of Clifford chemistry, but I don't feel like I've written up enough of the foundational stuff to explain things. The quick version is that the equivalent of cyclopentane should be able to avoid bond strain, which probably has some kind of implication for the equivalent of ribose, but I'm distracted by the possibility that there's potentially an allotrope similar to diamond, but with no direct three-dimensional equivalent. Or possibly there's no way to extrapolate the relevant structure outward, and it's just a weird crystal defect. Or maybe it's simply a substituent in a more complex crystal structure.

I need to find someone to discuss this stuff with. Later. For now, sleep.

Good night.

Weekly Roundup 2025-03-11

2025-03-11T04:00:00-04:00

Wednesday: I committed to ordering some packages.
Thursday: I scoped out some prep work in order to feel more comfortable about ordering the packages.
Friday: As part of that prep work, I poked around with Jujutsu a little. Still not sure how I feel about it.
Saturday: I purchased things besides the packages, and that was a little spicy for my credit card, because apparently buying a bunch of things on Bandcamp on the first Friday of the month looks weird no matter how often I do it.
Sunday: I ordered the packages, and also went through the day in kind of a haze.
Monday: I implemented balanced ternary integers in Python, because that's a thing you can do.

Next week, I'll see what I feel like working on; it'll be a few days before I have the printer, so I can't mess with it yet.

Coding 2025-03-10

2025-03-10T04:00:00-04:00

Okay, it's pretty late now, even if my body isn't feeling it yet, so I'm just going to spend a sentence or two on stuff.

I just implemented tritwise operations on balanced ternary, on a whim, and boy is it ever confusing. In addition to AND, both OR and XOR have the ability to mask out data, and that's just... too confusing for me to think about right now. (Morning edit: I wasn't thinking clearly; it's only XOR that can do this; OR can't, but also neither can AND.)

Anyway, nothing else productive is going to happen tonight. I need to get as much sleep as I can.

Good night.

Diary 2025-03-09

2025-03-09T05:00:00-04:00

I finally sent in the orders for everything I was thinking about. It'll be a few days before they show up.

Aside from that, I had kind of a weird day, and I'm hoping I have an okay night after it. I'm not sure whether I fully woke up before dinner, which is... unfortunate.

I did manage to get some stuff done, and I hope it pays dividends later, but we'll see. For now, I've got at make sure I get enough sleep, because I've got surprise plans for tomorrow, and they get started early.

Good night.

Coding 2025-03-08

2025-03-08T05:00:00-05:00

I just got through a bunch of internet purchases, which messed with my credit card, which left me a little keyed up.

I'm still making sure I have a proper plan for stuff to do with the thermal printer. And dealing with the fact that "mess around with heraldry hatching or Atkinson dithering" is a means, not an end.

Just going to run through my ideas here quickly. There are a few somewhat overlapping general concepts I'm hoping to take care of with all of this:

Rapid prototyping
Scratch paper with templates for specific purposes
Printing small graphs for offline reference
Creating physical artifacts, albeit cheap and compact
Generating diagrams, then manipulating them physically

I think these ideas need to bake a little bit more before I'm ready to place my order. (And hopefully by then the credit card will have recovered.)

Anyway, I spent way too long buying stuff, and I really need to get to bed now.

Good night.

Coding 2025-03-07

2025-03-07T05:00:00-05:00

I picked Jujutsu for now, and I don't fully have a handle on what workflow I want, but I've got the beginnings of something.

It turns out that "doing stuff with XML" is a pretty broad field, and "parse out Leo files and barely think about the XML" is pretty different from "generate SOAP payloads". I haven't generated a full payload, but I've put together enough components that I'm confident that I can smash something together with lxml, if the SOAP-specific libraries don't work for some reason.

With a bit more tinkering, I've got a good-enough handle on the software side of things; getting this to work the way I want is likely to be a project, but it should be a manageable one. With that out of the way, the next step is to make sure I know exactly what I want to buy, and exactly what the various third-party sellers are pitching me. Once everything matches up nicely, I'll put in the order, and wait, and possibly need to go to the UPS store for pickup because I don't trust them to actually deliver anything expensive.

Regardless, there's nothing more to do tonight, so I'm going to wrap things up.

Good night.

Coding 2025-03-06

2025-03-06T05:00:00-05:00

Okay, here we go. I've figured out, I think, which model of printer I want, but I'm planning to check over things a little more, because I was initially going for a model with features I didn't need.

Before ordering, I need to evaluate my ability to handle the SOAP data, and I'm doing stuff with Python for that, and that means getting a new project set up, and that means familiarizing myself with uv, and figuring out whether I want to track the files in Mercurial, Pijul, or learn Jujutsu.

Can't you use—?

You know I won't do that unless I'm getting paid.

Anyway, I'm tired and I want to wrap up, so I'll get back to this later. For now, bed.

Good night.

Diary 2025-03-05

2025-03-05T05:00:00-05:00

Well, I didn't really touch much today. I did some research for hobbies and projects that I'm thinking of getting into.

For the hobby, locksport. For the project, I've wanted for a while to get a thermal printer for stuff like making custom scratchpads, or printing out cards on the very, very cheap. (Sleeves would be basically mandatory.) I've basically decided what I want to do for getting started with locksport, and as far as the thermal printer, I've got a few models picked out to look into; before I order anything, I figure I should make sure I can use lxml (or whatever) to generate some of the code samples in a nice fashion. I'll see if I have the focus to do that tomorrow.

For now, sleep.

Good night.

Weekly Roundup 2025-03-04

2025-03-04T05:00:00-05:00

Wednesday: I started talking about some of the ideas behind Hivelets.
Thursday: Trapped behind potato internet.
Friday: Trapped behind potato internet.
Saturday: I considered how I should probably get more experience before trying to make serious progress on Hivelets.
Sunday: I wasn't able to do much.
Monday: I was able to do stuff, but it wasn't really focused.

Next week, I'm going to try to make some more progress on the stuff I was doing yesterday.

Diary 2025-03-03

2025-03-03T05:00:00-05:00

Weird combination of "unfocused about what I'm working on" and "seemingly accomplishing things". My current focus appears to be mainly on the Pettish "player emulator"; I'm trying to rewrite it into a form that makes more sense to me, and then I'll try either using it with other people's games, or to do some (probably not very informative, but I'm open to being surprised) testing against the jam version of Demiurgent Business.

Aside from that, I've once again got a weird urge to acquire a thermal printer and use it for random one-off things. However, I don't have more to say on that right now, because it's late.

Good night.

Diary 2025-03-02

2025-03-02T05:00:00-05:00

Okay, something in the air was off today, and I decided to just kind of take a break from screens. I thought some about writing that I'd like to try, but it was hard to get much done when I just wanted to lie down. Maybe tomorrow I'll have stuff together to try, or maybe I'll be still taking it easy.

Anyway, I let things get late again, so I should wrap it up.

Good night.

Diary 2025-03-01

2025-03-01T05:00:00-05:00

And I'm back! I technically could have posted while on potato internet, but I chose not to, because I knew using it would be extremely frustrating, and I just didn't want to bother.

While I was waiting for the internet to come back, I did some reading and thinking about what I should do for Hivelets. While I'll happily bash out lore or character class concepts, it's highly unlikely that I'll get where I want to with Hivelets without getting some experience with a more modest scope of project.

I've got various other things I want to mess with, and I did download/install stuff for them just now, but I'll have to wait until tomorrow to actually mess with them. For now, I'd better wind down and get to bed.

Good night.

Hivelets 2025-02-26

2025-02-26T05:00:00-05:00

I'm tired and not thinking too clearly, so instead of bashing my head against MOTR again, I'm starting a new category.

For a while now, I've been poking at an RPG concept. I don't fully remember some of the links in the creative chain, so I'm going to try to just devise an elevator pitch:

"Weird bug aliens infiltrate humanity, get entangled with people's everyday lives, and probably experience some form of gender feels."

The general idea is that this game, Hivelets, is based on Lancer, but instead of the key question being "are you currently piloting a robot?", it's "are any weird alien bits visible?". The other major difference is the standard roll, which is inspired by PbtA, and when I tried to describe it to my wife, her immediate point of comparison was Oblivion's "speechcraft" mechanic, and, um, hopefully it ends up making more sense than that.

There are two major areas that I need to focus on. One is coming up with enough different "weird bug alien" concepts to fill out the character creation/progression options. The other is understanding Lancer beyond "I read some blog posts, and read the rulebook for the lore and pretty pictures".

To be fair, the pictures are very pretty.

Anyway, I'm totally out of it for the rest of tonight. Going to wind down early.

Good night.

Weekly Roundup 2025-02-25

2025-02-25T05:00:00-05:00

Wednesday: I got a new matrix operation written.
Thursday: The matrix documentation has problems.
Friday: I fixed one of those problems.
Saturday: Pain.
Sunday: I started identifying specific issues with the matrix module.
Monday: I reviewed the matrix module to find problems, although some of them were basically just "the vibes are off".

Next week, I'm going to try to take a break from this and focus on some other projects for a bit.

Coding 2025-02-24

2025-02-24T05:00:00-05:00

I have the option to just post basically nothing and call it a night, and there's a decent argument that I probably should, but I don't want to. Let's have a look at the matrix operation names so I can try to figure out how they should be named.

@matrix: this one is fine actually
flatten(): this combinator removes a layer of Facts; my gut says it should be called flattened() (I'm not calling it join())
from_box(): aside from anything else, this is a bad constructor name, because it does not make a Matrix from a Box, but instead it retrieves a value from the Box; perhaps draw_from_box(), or maybe drawn_from_box()
from_selection(): this has almost exactly the same problems and presumably solutions
set_defaults(): recently renamed from include_box(), which is an improvement but I bet more can be done; with_defaults() perhaps?
map_over_1_matrix() and map_over_2_matrices: the @matrix decorator handles the other cases; the obvious change here is to go from map to mapped, but that doesn't feel like it gets to the heart of the issue, somehow
no_axes(): should really really really be something like constant_matrix(); I think this particular bad name is a legacy of when Matrix was called Parametric, and saying something like constant_parametric() made my teeth itch
reduce_axes(): aside from the tense issue, the code is insufficiently commented/documented, and makes my brain hurt (also, I'm not calling it fold(), and I'm probably not calling it folded())
shield(): new and untested, the purpose of this combinator is to prevent a single-value Matrix from being split into a multi-value Matrix if its Axes get product()ed with another Matrix (How does it do this? Kind of inefficiently, but in a correct manner.)

Well, there are a few ideas in there I can act on, but I'm still missing some key insight. I'm not getting it tonight; let's call it here.

Good night.

Coding 2025-02-23

2025-02-23T05:00:00-05:00

I'm still having trouble having a really relaxing day, but hopefully I'll get that tomorrow. For now, I want to get something done with MOTR, even if that something is just reorienting myself for future work.

Hm, yes, the next thing to do is to figure out tests for the new shield() operation, and then work on the "dependency" compound operation.

One of the other things I want to do in this area, though is to come up with generally better names. There are a few issues:

shield() was just come up with on the spur of the moment because I had to call it something
The other operations have kind of a... tense problem? They're declarative transformations of future computations, but they're kind of named from the perspective of the future computations, instead of in terms of the actual concrete values that the functions act on.
Finally, the different sets being tracked by the Axes objects just have some confusing names.

Anyway, that's stuff to focus on later. If I don't get out of the light and lie down pretty much right now, I'm going to have a much worse time than I'm currently having, and I'm not doing great at the moment.

Good night.

Diary 2025-02-22

2025-02-22T05:00:00-05:00

It's been a rough day. I don't have much to talk about right now. I need to get to bed.

Good night.

Coding 2025-02-21

2025-02-21T05:00:00-05:00

Okay, I'm not feeling great right now, but I want to try to get something done. I was looking over the matrix code earlier, and I noticed that there's a concept that can have a better name. It's called "include box", but in practice, it's specifying default values for some arguments. So let's try to get it to have a name that reflects that.

...

And with half an hour or so of work, it's done. Good job, me. Get some sleep.

Good night.

Coding 2025-02-20

2025-02-20T05:00:00-05:00

I'm looking over the Matrix code again, and coming to the conclusion that I need to review and revise the documentation. There are a bunch of ideas in there about set operations, and it's not obvious what I'm trying to guarantee. Also, some stuff is either counter-intuitive or wrong. Furthermore, I'll need to figure out what kinds of tests to write for the new code, and write them.

I don't have the time or energy for that right now, though. I need to wind down.

Good night.

Coding 2025-02-19

2025-02-19T05:00:00-05:00

Oof, I'm tired now. But I've put together a prototype for a matrix primitive that is the missing piece needed build up matrix dependency tracking compositionally. The hope with such a thing is that I can keep the scope of each individual operation down, verify each one independently, and then toss them together, rather than try to make a bespoke combinator that does exactly what I need, and then get it to work correctly.

Of course, I still need to make sure that this tiny thing I just drafted both works right, and can be used as a component to create something that actually accomplishes what I want. I'll try to check over those things tomorrow. For now, I'm really really really tired.

Good night.

Weekly Roundup 2025-02-18

2025-02-18T05:00:00-05:00

Wednesday: I had some thoughts about representing things that are conceptually adjacent to cards. (I still don't have a plan for actually using any of this code.)
Thursday: I got stuff done on that, and tried to figure out what all of the non-language bits of Impliciula will need to look like.
Friday: No energy.
Saturday: Again no energy, I think.
Sunday: I had some ideas that I worked on, but they didn't yet produce anything worthy of a blog post.
Monday: I got some work done on MOTR, and I think I'm starting to understand what was making me uncomfortable with this update.

Next week, I'm going to continue making reasonable changes to MOTR. And messing around with other things on the side.

Coding 2025-02-17

2025-02-17T05:00:00-05:00

Okay, so. I've laid the groundwork for tracking matrix dependencies, but I haven't yet made the decision for exactly how to implement some of the details. Like, okay, I need to track dependencies in the Axes, and have the correct logic for propagating dependencies up, along with the new "injected values" concept, but once I have support for that in the matrix module, I'm not sure what the sensible way will be to call that new functionality in the artifact module.

I'll see if I can get something done with this tomorrow, but for now I should be getting ready for bed.

Good night.

Diary 2025-02-16

2025-02-16T05:00:00-05:00

I got a bit done today. I got inspired to take my writing in a slightly different direction, so I'm trying to lay the groundwork for that.

Aside from that, I just kind of hung around. Maybe tomorrow I'll finally psych myself up to work on MOTR again, instead of psyching myself out.

We'll see. For now, I let things get so let that I have to wrap things up as soon as I can.

Good night.

Diary 2025-02-15

2025-02-15T05:00:00-05:00

I once again didn't get much done today. I'm hoping to get some writing or something done tomorrow, but we'll see.

I really would like to have something more to say here, but I just don't see how that's happening.

Good night.

Diary 2025-02-14

2025-02-14T05:00:00-05:00

I didn't have the energy or focus to work on any hobby stuff today, so...

Let's see how I feel tomorrow, I guess.

Good night.

Coding 2025-02-13

2025-02-13T05:00:00-05:00

Still psyching myself up for MOTR work. In the meantime, I implemented indexing unordered pairs of numbers via triangular numbers, so that's neat. It makes dominos work, and all. Not the pizza chain, the concept of... you get it.

Aside from that, I decided that, since the theoretical side of Impliciula is a bit intimidating currently, I can work on defining the tooling requirements. Some of the things that for many people wouldn't even be choices, have me making somewhat niche decisions.

You seriously looked at Atlassian and went "Hmph, I don't know you."

All I'm saying is, if the Jujutsu backend gets recommended, and eats git's lunch so hard that GitHub switches to it, I will have an improperly low amount of sympathy for anyone unhappy with that course of events.

Anyway, I need to get to bed several minutes ago.

Good night.

Coding 2025-02-12

2025-02-12T05:00:00-05:00

I wasn't quite ready to face the next rework that MOTR needs, so I spent a bit of time noodling with some code that I hadn't realized that I hadn't touched in almost a year.

(I'm sure I remember working on a more recent version, but rg can't find it, so...)

This project is around modeling cards, and I recently realized that there are things that share "logical structure" with cards without adhering to literally the same form factor. This means I've got a few things that I'd like to implement that shouldn't be terribly difficult with what I have (like mahjong and related sets of tiles), but I was thinking today about dominos. They've got a pair of ranks, and usually are treated symmetrically, which means that they need a different indexing operation than I've implemented before, to reverse the triangular number and get the, um, remainder. To get this to work, there's something about adding 0.25 and taking a square root and adding or subtracting 0.5, but I don't want to rush into implementing it before I establish some basic expectations.

Anyway, I meant to get to bed sooner, and I'll have problems if I don't do so soon, so I'm cutting this off here and now.

Good night.

Weekly Roundup 2025-02-11

2025-02-11T05:00:00-05:00

Wednesday: I thought I was about to write the coverage wrappers.
Thursday: Fuck Trump.
Friday: I wondered about whether to make a change in the name of "consistency".
Saturday: I started to get some idea of what stands in the way of just writing the coverage wrappers. I didn't have enough of an idea though.
Sunday: I worked out the two things I wanted done before anything else, and posted them in reverse order for some reason.
Monday: I got one thing done, and started freaking out a bit about the other.

Next week, either I'll summon the resolve to start on these next rewrites, or I'll switch to something else for a bit.

Coding 2025-02-10

2025-02-10T05:00:00-05:00

Okay, I got the SelectionDirectory tweaks in, and finally started to meaningfully cut down on the size of client code. It's just a few lines, but it scales with the number of commands, so that's a good sign.

While I was working on this, I realized that I'm going to want to rework the client interface in a major way that should make things generally nicer, at the cost of an extension interface that I don't feel super enthused about. I'm going to hold off on implementing that for now, though.

What's up next, hopefully in the next few days, is to rework the Input type into two separate types. The end goal is to make the Input type capable of tracking its source Matrix. Because a single Input can be stored in many Matrix objects, this is going to have to manifest in the form of a helper function that gets used by the new Link class.

Once I've got feature parity but with Inputs constructed dynamically on retrieval, I can work on implementing dependency tracking in Axes objects. Once that part looks to be in a good shape, then I can add dependency tracking to the Input.convert() method, which should allow for drastically simplifying the interface.

The obvious next thing to do would be that big rework, which I should do before messing with the coverage wrapper, but maybe I'll try to work on some other project for a bit instead.

Anyway, it's late and I should get to bed.

Good night.

Coding 2025-02-09

2025-02-09T05:00:00-05:00

Okay, let's see where my plans have ended up.

Because Artifact instances work by constructing a bespoke Matrix, it's not easy, but straightforward to handle encoding dependencies into an Input. One subtlety is that OutputFromInput is going to need to be passed an Input with depedency data, and that data needs to be correctly propagated. Or... hm. Either I allow arbitrary dependency association, or I do the injection implicitly, and have OutputFromInput specifically take... Ugh. It's actually kind of hard to confirm that a given Input actually "came from" a given Matrix. I think I can still do it, but it will be... fascinating.

Encoding Matrix dependencies on the Axes looks about as straightforward as I originally thought tracking them on Input would be, and handling the dependencies in the facts call should be no problem.

Now, let's see about moving some client code up to remove duplication. What I'm basically hoping to have work is, to create a wrapper around SelectionDirectory, which supplies values for the associated selections. Tracking these would require more Axes tweaks, but they would be pretty mild.

I was resting and planning all day, so I don't have time to put these into practice tonight, but I'm now reasonably confident that making the changes will work. First SelectionDirectory, then Input. Sadly, I think I won't actually need the context-variable rewrite, so I can/should hold off on that until I get more coverage.

Anyway, it's late and I should wind down.

Good night.

Coding 2025-02-08

2025-02-08T05:00:00-05:00

I was pretty tired again today, so I focused mainly on planning, with a side of addressing some Mypy errors that slipped through the cracks earlier.

The conclusion of my planning was that, once I address the various flake8 issues that I've introduced, I can enter a commit, and get to work on other improvements that this latest effort has revealed as possible.

Those look something like:

Determine whether it is feasible to track additional Box entries on Matrix Axes. (If not, what alternatives exist? If so, is it still necessary to provide a facility for dynamically injecting such data during matrix evaluation? (Looking ahead to my future plans, I think it is necessary, because injecting command-specific data in a shared label would interfere with sequencing operators.)
Replace passing Box and Selection objects with context variables.
Track additional Box entries on Matrix Axes.
Move specification of command-specific constants out of the compendium construction function, into the matrix creation code.
Create sequencing operators for Matrix objects.
Track Matrix dependencies along with Input objects derived from a Matrix.
Automatically sequence linked Matrix objects.
Create a helper object for working with Python-specific Matrix objects etc.
Finally write wrappers for coverage.py.

After that, hopefully a release won't be too far off, but we will see.

Good night.

Coding 2025-02-07

2025-02-07T05:00:00-05:00

I have been planning things for MOTR. Various new bits of code that is sort of monadic or whatever, improvements to wrapper authoring and use, and some fancy implementations that I think are long overdue. Before I do any of that, though, I still need to clean up the changes I've been making, and to finish that up, there are a few things I need to consider.

Basically, I've got a helper class for some wrappers, and I'm trying to figure out whether a parallel construction would make sense for the remaining wrappers. I haven't figured this out yet, and I won't figure it out tonight. Maybe tomorrow.

Good night.

Diary 2025-02-06

2025-02-06T05:00:00-05:00

I mentioned the inauguration finally sinking in. I think part of that is the pace of news and the constant back and forth.

There's an agricultural crisis that will make things harder on the American consumer, exacerbated by Trump's planned tariffs. Wait, never mind, about the tariffs, not the crisis, though, that's still happening, I think. The tariffs are off the table, thanks to Trump's trademark negotiation tactics. Self-proclaimed free speech advocates are censoring government websites. No, that's getting rolled back. Somewhat? Elon Musk is somehow the most powerful man in America now. ... Probably don't fly.

It's like a kaleidoscope of horrors and faint hope, and I don't even know if anything I just said is or was ever accurate.

I don't generally say much about politics on this blog, and that's because I generally don't have much to say, and I feel like I shouldn't. Like the young men Musk has gotten to help him strip the country for parts, I've got a focus on STEM fields, but unlike the stereotypical STEM bro, I've got some idea of how many sub-fields there are. I'm paper-qualified in one area of engineering, and self-taught in another. There is so much of STEM alone to which that knowledge does not generalize. So why should I have anything to say about policy decisions?

Except. Except that there is a culture war on, to try and stop people like me and my family and friends from existing. To tell us how to live our lives, in the name of freedom. To turn us into a boogeyman, when we just want to hold down jobs, make art, and watch movies or whatever.

There is so much to be done to deal with this, but part of it is just to be there for myself and the people in my life. I know there are more people out there reading my blog than I can quite understand how you all got here. I want to thank you all for reading what I have to say, and if you're feeling as scared and uncertain as I am, I want you to find the people that matter to you, and make sure to be there for them.

I'm dealing with a lot right now, and I don't know how much more I have to say on this, so the "grease-monkeying around inside MOTR" content that's been so much of my output lately is probably going to start back up tomorrow.

Good night.

Coding 2025-02-05

2025-02-05T05:00:00-05:00

I may or may not be at a good-ish stopping point for these MOTR updates. I've got some ideas I want to follow up on, but I also need to take stock of the state of the wrappers, and get motr back to mostly passing.

Once I've got that done, I'm going to work on writing the coverage wrapper, and see if that finally drives my missing/partial counts below 31/7.

Oh boy, I got distracted trying to figure out what's going on with some of the coverage gaps. I have filed some weird bugs over this code. Anyway, I need to wrap up.

Good night.

Weekly Roundup 2025-02-04

2025-02-04T05:00:00-05:00

Wednesday: I made some improvements to MOTR.
Thursday: I found more changes to make to MOTR to hopefully improve the wrappers.
Friday: I came up with a name for the newly-developed concept.
Saturday: I put the name change into practice.
Sunday: I used it, and then found another case of "these two things should be put together".
Monday: I iterated some more on the designs around all of this high-level stuff.

Next week, I'm going to try to take stock of my changes and tidy things up a little. At the same time, the inauguration is finally sinking in for me, and I'm going to have to see about doing something about *gestures vaguely*. I've got some small things I want to put together, but I also need to figure out how to support more substantial opposition.

Coding 2025-02-03

2025-02-03T05:00:00-05:00

Well, I'm currently up to my elbows in MOTR's top layer of abstraction. I'm following hunches to eliminate repetition, but I'm going to have to follow a few more hunches to eliminate verbosity.

The big thing that's happening is that I'm going all in on the pattern of writing decorators to convert one-off functions to constant values. Not only am I using this to generate the base commands, but I'm also applying it to the wrappers for --command-arguments.

I'm feeling generally good about these changes from a module layout perspective, but I need to be considering whether to mess around with the naming conventions. Because it's... somewhat questionable to have some constants named in lowercase because of Reasons.

I'm going to wrap up for now, but I've got to finish up with these updates, and ponder the naming.

Good night.

Coding 2025-02-02

2025-02-02T05:00:00-05:00

Guess what time it is again!

I don't want to...

That's right, it's time to try to come up with a name for a concept that I just discovered a few hours ago, and have yet to properly articulate!

:) (glitchy)

Why...

So, I'm right now trying to increase the prominence of "Boxes" in MOTR's higher-level code. Like, you call a function, you get back a box, you can stick them together, maybe I write a helper for that to complain if you try to reuse a label, it's all good.

And when I was working on this, I realized that at one point in client code, the user is expected to construct Matrix objects from Command objects, but in order to actually get those matrices to generate Fact objects, it's sometimes necessary to augment the basic Box (constructed by other helper functions) with additional assignments, using information contained in a python_helpers.EntryInstaller that is next to the Command. When this is required depends on the nature of the Command object that was used for each Matrix. This means that there are these implicit connections that I once again want to make explicit.

I want to create an object that contains the Command and the EntryInstaller. I know this would go in python_helpers, and I have a decent idea of what the constructor should look like. The goal here is to provide easy access to the parts, for wrapper code, and to provide decent idioms and helper functions for client code.

Naming, though...

One important thing to note is that one of the module-level variables which, in only these five instances, I intend to replace, has a different name than the others. This represents different type parameters, and should be reflected in the variable name of the new object...

With that in mind, I think I could get away with the very on-the-nose InstallerAndCommand.

Introducing InstallerAndCommand after ExecutableAndInvocation? This is getting out of hand! Now, there are two of them!

Anyway, I'm not committing to this until I've had some sleep.

Good night.

Coding 2025-02-01

2025-02-01T05:00:00-05:00

I was out of it for various reasons today, and then we had a not-great drive, so really all I accomplished today was to rename "FooPart" to "Link".

I'll take it, and tomorrow, I'll work on actually using Link to write and consume wrapper code.

Good night.

Coding 2025-01-31

2025-01-31T05:00:00-05:00

Hm. The code to do what I want is very simple. Except.

Except for naming the class. The point of this class is to replace pairs of variables. One variable is a label that is used in the construction of the other, a command part. When the command part has been used in a command, and that command has been converted into a matrix, the label can be used to retrieve an input artifact from the matrix. This is a key operation in chaining commands together.

Hm. Chaining.

Perhaps this is a "link"?

I'll have to think about it.

Anyway, it's late and I need to wrap things up.

Good night.

Coding 2025-01-30

2025-01-30T05:00:00-05:00

Okay. I took another look at my test code, and identified an area of potential improvement. Basically, there are some places where two variables are closely related, but this is only indicated by "thematic connections" in the variable names. Not good enough. The related values should be accessed through a single object, rather than next to each other in module scope.

From a typing perspective, I cannot "just reach into" one object and retrieve the other; there are too many layers of polymorphism/erasure. So, I need to create a new object to hold both values. I'm thinking a class with a constructor that takes a label (one of the object's types) and returns a decorator that wraps a function taking a label of the same type, and returning a command.Part (or possibly multiple?). This would have the additional advantage of associating type information that wasn't available before because of Reasons.

I was pondering before whether to make this generic (in an informal sense—it has to be generic in the label type, and being generic in the program type would be nice), or specific, and I'm thinking right now, I'll try making it specific. That will let me do some things that make the end-user interface more compact, which is a definite plus.

Anyway, I think those are enough notes to write this later. I'm sleepy now, so I'm going to wind down.

Good night.

Coding 2025-01-29

2025-01-29T05:00:00-05:00

All right, I'm not putting quite as much time as I'd like into MOTR, but I got one of the obvious improvements to the interface made. The next thing I think I should consider is moving some code out of "test helpers" and into the MOTR package, and then figure out how to improve it.

This code is used to associate particular programs with the Python versions they should be run against. When I get the coverage wrapper written, I'm going to have different objects that all refer to the same version label, and the user really shouldn't specify a version set more than once per program. I'm not quite sure how to handle this; I'll have to think about it for a while.

For now, sleep.

Good night.

Weekly Roundup 2025-01-28

2025-01-28T05:00:00-05:00

Wednesday: I had a good time reworking the top-level interfaces I'm designing for MOTR, and I found this somewhat surprising.
Thursday: I got a bit more work done, and did not write it up well. I could check the commit logs, but I don't wanna.
Friday: I got work done, then decided to take a break to rethink things.
Saturday: I figured out the general tack I wanted to take with the next round of improvements.
Sunday: It worked, so I started thinking about writing more wrappers.
Monday: I remembered that getting the user interface right is also a thing I want to do, and while I've probably made things better in that area, there's certainly more to be done.

Next week, I'm going to try to work through those improvements, and maybe pick up another project.

Coding 2025-01-27

2025-01-27T05:00:00-05:00

Okay, we traveled a bunch and then I just veged out for a bit, so I'm just going to jot down some thoughts I had on the drive back:

I'm not quite ready to write the coverage wrapper.
In particular, I need to look at this code from the perspective of an end user.
Are there any pain points in the current interface?
My planned design for the coverage wrapper will introduce some potential pitfalls into the interface. Can I write a helper function that is more pleasant to use than the current interface in that area, but can error or warn (as appropriate) when the user approaches those pitfalls?
I've forgotten some of the plans I made for handling the erase-run-combine flow. Can I rediscover or redevelop those plans? (I believe I can get most of the required functionality by just writing a function that takes a value of the approriate type, but we'll see.)

Anyway, I'm pushing things way too late, better get ready for bed now.

Good night.

Coding 2025-01-26

2025-01-26T05:00:00-05:00

I put the changes I was talking about yesterday into practice, and now I'm looking over the state of the wrapper code, and... And I think I'm finally ready to write the coverage wrapper.

Hopefully, writing the coverage wrapper will help me shake out some of the weird gaps in current code coverage, but even if it doesn't, it should get me one step closer to cutting a release and merging the topics.

In any case, we'll see about that later. I had a bit of a time these last few days, and I need to get ready for bed.

Good night.

Coding 2025-01-25

2025-01-25T05:00:00-05:00

Okay, let's think a little more about the API improvements that I want. The basic point of confusion I'm hitting is in the design of some classes for wrapping Python packages. Basically...

It is often the case that a Python package that installs a script or a runnable module will have the same name as that script or module. From a reporting perspective, it makes sense to put the output of a script into a distinct directory, and the name of the script is usually a good candidate for the name of the directory.

So, given a script/module, by default we want to install the package of the same name, and put output in a directory of the same name. There are several points of possible variance here:

The package requires extras to be specified.
The package requires a version range.
The package has a different name.
Additional packages need to be installed.
The output should go in a directory with a different name.
The output should not go into a different directory. (This is simplified from what actually happens, which is a little fiddlier to explain, but should be intuitive in practice.)

Thinking about this, I'm kind of tempted to make everything run off of str.format(). The requirements should go in the object, and I'm not sure whether the directory should go in the object, or be specified in the call to convert the object to a form legible to the lower layers of abstraction.

I think this gets the bulk of the planning on this taken care of. Hopefully, I'll get the rest of the details worked out soon.

For now, I'm going to wind down.

Good night.

Coding 2025-01-24

2025-01-24T05:00:00-05:00

Okay. Top-level wrappers are somewhat simpler. There's more to be done, but I'm not sure how to go about it yet, so I'm just going to let it stew for a bit.

Basically, I wrote some of this helper code with somewhat strange semantics, and I'm trying to figure out how to make it do things when it should, and not do things when it shouldn't. More or less.

I don't want to think about this any more right now, though, so I'm going to get ready for bed.

Good night.

Coding 2025-01-23

2025-01-23T05:00:00-05:00

I got the initial pass on updating the wrapper APIs done, although I had to add some new capabilities that I missed.

However, there's more work and open questions to address:

I only got the code to typecheck, so I know that, at minimum, I broke the tests.
I still want to write some new helper methods slightly lower in the stack.
Some of the new interface is not quite right, so I want to modify it and see if I can get it to work in places that the old interface didn't.

I'm still feeling good about these changes, because they generally resulted in shorter code at the higher levels of the stack. Once the current round is really done, I'll see if I'm up for writing more wrappers, which will hopefully help me increase my coverage.

Anyway, I need to get ready for bed.

Good night.

Coding 2025-01-22

2025-01-22T05:00:00-05:00

All right, I have some planned work that I think I need to put off until I see how it's actually reasonable, but I'm putting some of the work I've done so far into practice, and it... seems reasonable? Somehow it inspired Mypy to notice or invent new typing issues, so that's cool, but I think that problem will go away when I finish the rewrite.

Basically, I'm discovering that I kind of have "the wrong interface" for putting together command-line arguments, and I'm currently writing "a better interface" over "the wrong interface". Once I have everything going through the better interface, I'll try to cut down on the wrong interface. Fortunately, this is at the very highest level, and I've only written a handful of consumers, so these rewrites are pretty cheap. If I get good vibes from finishing the refactor, then I'll commit to using the new interface to fill out the rest of the wrappers.

It's also possible that I'll end up adding more helper methods, but I'm not totally sure yet what's going to look good.

Anyway, I really really really need to get to bed.

Good night.

Weekly Roundup 2025-01-21

2025-01-21T05:00:00-05:00

Wednesday: I had a general idea of generating LLVM IR from the STLC.
Thursday: I ran into trouble figuring out how to avoid undefined behavior.
Friday: I had some ideas about that, but I don't know if they're good ones.
Saturday: I tried to be laid back, in part to be ready for doing something in the morning.
Sunday: (I didn't mention it, but said thing was actually planned for next week. Oops.) I stepped away from Impliciula, and messed around with Mythic rolling and improving MOTR.
Monday: I got just enough done trying to improve MOTR to convince me that I'm on the right track.

Next week, I'm going to finish up the MOTR refactor, and try to make progress on general writing stuff.

Coding 2025-01-20

2025-01-20T05:00:00-05:00

Okay, let's go, and make this quick. I was kind of spaced out today, so I'm going to look over yesterday's post, and try to quickly throw something together to improve things.

...

Okay, I've got code together that should allow me to clarify some common code patterns in these modules and reduce boilerplate, but I'm not comfortable going ahead until I figure out the right amount of boilerplate to get rid of, because sometimes the code isn't quite the same between modules, and I'm looking at it, not totally sure if these differences are meaningful. Like, I'm going to need to, at a less sleepy time, go over this code and figure out what the actual flow is, because I've totally forgotten most of what I was doing, and the stuff I do remember seems a little questionable.

Things to try to figure out later:

Can I turn new_typical into a keyword-only argument to new?
Does it break anything if I add a Placeholder pip object with nothing in it to the Python wrapper code? (Is its current absence breaking anything?)
Can I add a helper method to runner.ExecutableAndInvocation to be the equivalent of the runner() method, but it instead produces a ModuleRunner?

By pondering these questions, I have gotten... somewhere, and I should be able to get a little further in the days to come. For now, it's later than I'd like.

Good night.

Coding 2025-01-19

2025-01-19T05:00:00-05:00

In keeping with letting Impliciula sit for a bit, I took a look at MOTR today, and messed around a bit more with the Mythic stuff. For the Mythic stuff, I just worked on the resource importing and processing side of the meaning tables. All of that is almost ready for the next level of abstraction; I'm just missing the special handling required for the name table, which I may actually simply do a layer up.

The MOTR side of things is... confounding. I'm trying to take some of the incomprehensible boilerplate I wrote, and render it comprehensible, and I'm having difficulties. Like, suppose you have a task that runs some command. That command is going to be available in some environment, as a result of some installer installing a package. Now, a package can make any number of commands available, and, hm, I'm not sure if this is even the part I should be focusing on.

Because I just noticed a source of regularity that may help me organize things. Most commands have to provide both a matrix.SelectionDirectory[PathWith[pip.VersionLabel]] and an installer.Label[pip.PipArgs], and a few instead only provide an installer.Label[Placeholder[pip.PipArgs]]. These correspond to semantic differences that I believe it makes sense to model with classes. Those classes can then refer to the program class, which should resolve the concerns I had while I was writing the previous paragraph.

Anyway, nice breakthrough, really need to get to bed soon.

Good night.

Diary 2025-01-18

2025-01-18T05:00:00-05:00

Okay, let's see.

I do have some thoughts about Impliciula, but they boil down to "I think I need to take a break and reconsider my approach". (Maybe this time I can go through a tutorial without inventing more work to do.)

I've got other things I want to pick up, but my top priority right now is to make sure I take care of some stuff in the morning.

I'm about ready for that, and afterwards, I'll work on... something.

Good night.

Coding 2025-01-17

2025-01-17T05:00:00-05:00

So, my focus has been questionable at best, but I am still considering how to prevent undefined behavior in a flexible way.

My hope is that I can attach a lot of information to the underlying address range, and some to individual pointers, using some kind of giant predicate that gets processed specifically with linear logic.

Let's imagine the simplest scenario that I suspect could totally muck this up:

We have two pointers to distinct address ranges, called a and b. a is stored in a third pointer, x, which has two copies. Next, b is stored in x. As a result, a is no longer accessible, and by extension, whatever it pointed to is no longer accessible. Conversely, b is accessible in its original location, as well as in both locations that x is accessible from.

One of the challenges I have to face is figuring out just how general I can make the general case. The idea I have is, it should be possible to take a pointer that nothing else knows about, stick it in a wrapper that provides desired guarantees and usage patterns, and then just go. Naturally, that bit requires substructural stuff to guarantee that the pointer can thereafter only be accessed through the wrapper. The whole point of such wrappers is that they provide different interfaces, so I suppose the design would be "there's no baseline behavior, rather a set of requirements from the various LLVM intrinsics, and implementing a wrapper requires somehow proving that you've satisfied those requirements". Requirements I have in mind include:

The pointer is based on the address range. (Okay.)
The address range is live. (All right.)
The pointer has the required permissions. (Should be straightforward.)
In the absence of synchronization, a pointer shared between threads can only be read. (... Oh no.)
Some stuff about aliasing in the context of single-threaded code as well? Like, sometimes, we'd actually like to have funky behavior from passing the same pointer twice, and sometimes we'd like to be prevented from doing so.

Thinking about this, it's possible that I've been misinterpreting the meaning of "synchronization" just enough to freak myself out. I'll have to look over the LLVM docs some more later. For now, I really should get to bed.

Good night.

Coding 2025-01-16

2025-01-16T05:00:00-05:00

Okay, this day was a bit weird, so I'm going to just dash off some quick thoughts, then post.

I've been reading and thinking really hard about how to handle address ranges, and the more I think about it, the more confused I get about how to do so.

I think what I'd like to do is to put together something with the simply-typed lambda calculus, and call-by-push-value. Only emit calls that can't trigger undefined behavior. Hopefully, doing this will get me some insight into how to handle addresses.

At the same time, though, I'm not sure how to actually handle emitting the IR. I've found some crates, but I'm having trouble understanding the documentation.

I'm going to wind down now as best I can.

Good night.

Coding 2025-01-15

2025-01-15T05:00:00-05:00

I've got a rough outline of the transformations I want to do to Impliciula code, with the general idea that LLVM IR is like the simply-typed lambda calculus, except that there are a bunch of things that can go wrong, which the higher levels need to deal with. (And there are modules, so, like, something involving existential types happened, I guess. And probably other things.) However.

Once I put the thought in properly, I realized that basic closure-based handling of resource lifecycles, while sufficient for many purposes, simply won't cut it for all use cases. So, if I want verifiably safe pointer manipulation that's also flexible, I need to also be reading up on substructural type systems, specifically affine types. My hope is that I can somehow connect this to refinement types so that it's possible to construct custom theories of pointer lifecycles. If I can't make that work, then I kind of need to reevaluate the whole direction I'm taking this project.

I have no idea whether I should expect that to work, but if it doesn't... ugh. Anyway, I'll have a better idea whether it will or not once I do some more reading, so I'm going to wrap this up now.

Good night.

Weekly Roundup 2025-01-14

2025-01-14T05:00:00-05:00

Wednesday: I got some tooling written for Mythic-related stuff.
Thursday: I made some progress on Mythic, and also thought some about Impliciula.
Friday: I laid out some daring ideas for Impliciula that might not pan out.
Saturday: There is nothing to say about this day.
Sunday: I wrote down some of the ideas I have for Impliciula that were kind of just... in my head, and I figured "Yeah, that probably makes sense."
Monday: I finished copying the tables over for Mythic.

Today, I found out that my teeth are somewhat healthier than they were six months ago, so that's cool.

Next week, I'm going to see about using these tables, and also about putting together a sketch of various things for Impliciula:

How effects and refinement types fit together
Subtyping and casting
Base types from LLVM
Refinement types required to avoid UB (the easy part is "this is non-zero", "this application doesn't overflow" is a little more interesting, and getting pointer provenance to work is kind of the whole point of this particular area of scope creep)

Coding 2025-01-13

2025-01-13T05:00:00-05:00

Okay, so I'm still looking over stuff for Impliciula, but I don't yet have anything to say about it.

Just now, I powered through the rest of the big tables for Mythic, so I'll be ready to write code to roll on those tables soon, and then to synthesize them into proper rolls.

For now, though, I need to wind down; I got suddenly really tired just now.

Good night.

Coding 2025-01-12

2025-01-12T05:00:00-05:00

All right, I've read up on effect types, and as I side, I'm interested in the theoretical implications of taking the paper's construction of them over a typed lambda calculus, and instead doing that above 1ML. I think the specific interaction I'm interested in is, what happens if I perform an effect in a function that returns a value of type type. But probably the more practical focus is, can I have different "top-level handlers" in different execution contexts, and then structure function calls something like [all type arguments] -> [evidence vector that can be evaluated at compile time] -> [actual arguments] -> [evidence vector that can be evaluated at run time] -> [return type]?

However, I don't want to rush into anything, so I'm back to reading up on refinement types. I'm not going too quickly, since I think I'm still feeling some of the effects of last night. I'm going to wrap this post up now.

Good night.

Diary 2025-01-11

2025-01-11T05:00:00-05:00

I'm in no shape to write about anything complicated right now, so I'm just going to take it easy.

I don't feel like saying any more.

Good night.

Coding 2025-01-10

2025-01-10T05:00:00-05:00

All right, I finished some reading on types, and now I've got more reading to do. To be clear, it is basically certain that I'm going to have to revisit this reading, almost from the beginning, but I do feel like I got some key insights from it. However, at this point, I think I really need to focus on two things: how to implement an effect system, and getting a feel for 1ML.

This is because... let me see if I can get this to make sense. 1ML attempts to unify the syntax for type-level and term-level functions. I think. It appears to me that doing so in the context of an effect system would give an implementation of implicit function arguments; those would be from effects that are handled at compile time. The failure mode for a typing algorithm is to fail to terminate. An effect system can (and in Koka, does) model the possibility of a particular computation failing to terminate.

With all of that in mind, the overall direction I'm trying to go seems like it has three possible endpoints, and I'm not sure which to expect:

It is possible to disallow non-termination at compile time, but still have expressive constructs, provided a non-termination proof is provided.
I somehow mess things up so badly that literally every compile-time function that does anything remotely interesting needs to be marked as potentially non-terminating.
It turns out that potentially non-terminating code is somehow not fundamentally any more interesting.

In any case, I would very much like to see how getting an effect system to work with 1ML (if that's actually possible the way I'm imagining it) stacks up against COCHIS.

Okay, that's all for now. Time to wind down.

Good night.

Coding 2025-01-09

2025-01-09T05:00:00-05:00

Well, after spending longer than I'd like debugging my laptop—

Ugh.

Do not ask. Do not take the bait. Do not provide the satisfaction—

I got back to copying tables over. On the one hand, it's almost done. On the other hand, exactly one of these last few pages secretly has a new, incomprehensible layout, and I need to psych myself up before I'm ready to either tweak my cleanup code again, or simply transcribe this one page.

So, putting that aside for now, I should get back to reading for Impliciula. One thing I need to put a pin in, even though I'm really curious about the tradeoffs, is how much expressive power I want to give the predicates in the refinement types. I really really really want to enable the pattern of "here is a simple implementation that is obviously correct, here is an optimized implementation, here is a proof that they give the same results on all inputs". (Okay, so technically it's better to prove it for given inputs without explicitly quantifying, because that's easier somehow. I think.) Looking over some of my own Dafny code, it looks like at least some of the stuff I want to prove doesn't need an excessive amount of power, unless I'm misreading things. (Also should allow things like "these related functions are specified with reference to each other's specifications, but their implementations are related differently, if at all.")

Anyway, time to wrap up, wind down, whatever.

Good night.

Coding 2025-01-08

2025-01-08T05:00:00-05:00

As I pointed out in the roundup, my initial plan for automatically reformatting tables was completely wrong for the majority of them, but I did manage to throw something together, which took the transcription process from "likely to cause significant harm to my mind, body, and soul" to "eh, pretty tedious". That's why I'm not done, but there's no reason I can't finish it up sometime when I have half an hour or so free.

Once that's done, I'll have to remind myself how loading resources works; I remember it basically making sense, but feeling somehow slightly less convenient than I'd like.

Back on the Impliciula side of things, I'm still reading up on types, but I'm also shoring up my knowledge of the stuff that I want that isn't covered by the textbook. I think the main thing I'm missing right now is stuff around continuations.

Right now right now, though, bed.

Good night.

Weekly Roundup 2025-01-07

2025-01-07T05:00:00-05:00

Wednesday: I read about type systems, and started rereading about Mythic.
Thursday: A bit more progress...
Friday: Some more progress, and pondering a large number of percentile tables.
Saturday: The percentile tables are being troublesome.
Sunday: I sketched out a setuptools plugin to avoid using MANIFEST.in files in Pijul repositories. (It's already possible to avoid it in the case of Mercurial and that other one.)
Monday: I thought I had a quick way to deal with the percentile tables; I have since realized that I was broadly incorrect.

Next week, I'm going to try to get that processing done right. Also, my skimming about type theory should come to a close, and I can start sketching stuff out and figuring out what to revisit.

Coding 2025-01-06

2025-01-06T05:00:00-05:00

Well. I just forced out a bunch of junky Python code. Later, I'll see about checking whether it in any way works, but first I have to decide whether and how I want to support the versioning aspect of setuptools-scm, in addition to the file tracking part. (I have to admit, I'm still smarting from dealing with Poetry's SCM integration however many years ago (I was doing something very weird.), but I'm hopeful I can get this to replace "maintaining a MANIFEST.in file" for the cases where I'm using an obscure SCM.)

I suppose you think it's admirable that you didn't crowbar in some way to pretend you have no idea what Git is?

I couldn't figure out how to—I mean, Git? What's that?

... Hey, wait a sec...

Anyway, I guess I need to psych myself up for transcribing a lot of information. Unless I can... Hm... Copy out an entire table. Split on newlines. Strip leading spaces. Leftpad all entries where the second character is a colon with zeroes. Sort. Remove the first four characters. Print.

Okay, yeah, that's no big deal to do later. For now, I want to get ready for bed, so I'm not ambushed by midnight again.

Good night.

Coding 2025-01-05

2025-01-05T05:00:00-05:00

Weird morning. Okay day.

I'm skimming the type theory stuff more aggressively, so it's good odds that I'll have to come back to this later, but I should have a good baseline to designing the different tiers of Impliciula. Have to handle stuff like resolving effects, and various kinds and levels of subtyping.

Anyway, I'm still calling this a Mythic autoroller post, so let's see what I need to do to get these tables in and accessible... Well, for the latter, I just updated the MANIFEST.in file, which, I'd kind of like to find an alternative.

...

Ah, I can write a setuptools plugin that interacts wit Pijul thus:

Given a path to a directory, run pijul list --repository {directory}, but also scan the ancestors of that directory looking for a .pijul directory. What we want is the relative path from the repository root to the directory that was passed in.
If we're given an empty string, just run pijul list, and scan from the current directory.
At this point, we have all of the information we need. We simply need to filter the list down to the entries prefixed by relative path we found, and return the rest of each such path. The "don't think too hard" way to do this is to turn everything into pathlib.Path instances and call candidate.relative_to(directory, walk_up=True) and filter out all entries that start with . or ...

Ooh, it looks like setuptools-scm provides some endpoints, so I could try doing things that way and see how it works.

(Note to future self: it seems that the tag in Pijul is associated with a state, which can be requested from the log. Comparing the state information should allow for determining how many changes have been recorded since the last tag.)

Anyway, it's late and I should get to bed.

Good night.

Coding 2025-01-04

2025-01-04T05:00:00-05:00

We traveled again today, so I'm a little out of it. That said, it's mostly nice to be back home.

Aside from that, I'm reading some heady theoretical stuff on the typing side of things, and I've come to a decision on laying out the percentile tables. I've transcribed two of them, but unfortunately, it looks like the first one was the easy one, and the second one and the remaining 47 are going to be a huge pain. So much so that I ended up drafting a custom table rather than transcribing any more. I'll publish that table somehow or other, once I think it's done.

For now, I really need to wind down. I'll see if I'm up for more of that type theory stuff.

Good night.

Coding 2025-01-03

2025-01-03T05:00:00-05:00

I've figured out the next thing I should be working on for Mythic. Unfortunately, it's a little involved; I'm going to transcribe 49 percentile dice tables, and aside from the effort that will take, I haven't settled on how I want to lay out the tables as files.

Aside from that, my reading on types has gotten to the basics of implementing object systems. I don't think this will be relevant for Impliciula, but it could be helpful for messing around in Lua, so I'll keep at it.

I'll get back to that reading now, and think about laying out data later.

Good night.

Coding 2025-01-02

2025-01-02T05:00:00-05:00

I'm learning some important stuff about type systems, and also I'm making some key progress putting together stuff for Mythic. On the Mythic side of things, I've got a few straightforward things to crunch through, and a bunch of data to enter. That should be done in a few days.

For now, I'm going to wind down and read up on types a bit more.

Good night.

Coding 2025-01-01

2025-01-01T05:00:00-05:00

I'm still looking over type and programming language stuff, but I'm also tossing together stuff in less disciplined fashion to try automating bookkeeping for Mythic. I don't know whether I'll actually need this, but writing it will be handy for refreshing my memory about how Mythic's second edition fits together. I mean, there's not much to it, but all the more reason to see if I can internalize it all.

I can talk some about the thought process that got me here. I've been trying, on and off, to get into tabletop roleplaying, but I really frequently have hit something along the lines of stage fright, so I'm trying to get some practice with solo roleplaying. (This is where the Stars Without Number stuff was going, although going into that, I wasn't quite aware of the amount of prep work my standards would end up giving me.) The last time I tried to do something in this are, I ended up with a character concept that I felt was too ambitious, and ended up not going anywhere with it. My current plan is to work with my wife as such: she handles most of the work, and I take care of the tedious bookkeeping. This gives me practice for bookkeeping on my own, and time to come up with a concept that's more in my comfort zone.

But for now, I'm just kind of slopping code together to see what seems to make sense. And there's only so much more of that I can do this year.

(Okay, so, see, I date these entries so that they correspond to midnight UTC the day after I write them, so...)

Good night.

Weekly Roundup 2024-12-31

2024-12-31T05:00:00-05:00

Wednesday: I put together a general outline for how Impliciula is going to have to work.
Thursday: I tried to figure out how to represent the core language for Impliciula.
Friday: I put together a sketch of code for an AST.
Saturday: I got a better idea of how I'm going to have to elaborate on the AST.
Sunday: I found a textbook on this stuff.
Monday: Reading the textbook is going well, but I'm slightly dreading the paper I'm going to have to write on Impliciula's type system.

Next week, I'm going to see if I can find something else to discuss.

Coding 2024-12-30

2024-12-30T05:00:00-05:00

Well, I'm somewhat winging reading the early chapters of this book, but it's fine, probably. I'm taking notes on the stuff I'll need to keep in mind later, and internalizing the correct interpretation of some notation that I was just sort of fudging my interpretation of.

I can't be sure that I'll continue to move through later chapters at the same clip as these early ones, but I'm just not going to worry about that right now.

What I am a little worried about is, it's talking about desirable properties of a type system, that I can't simply assume hold, and will need to prove. I'm a little worried that I may end up doubting myself just enough that I write a bunch of the compiler in Dafny, and then compile it to Rust, which is likely to produce code that is... somewhat odd-looking, at best.

Anyway, it's late now.

Good night.

Coding 2024-12-29

2024-12-29T05:00:00-05:00

Every time I read stuff on implementing type systems, I feel like the ideas being communicated are somewhat different. Hopefully this is a case of "there are many ways to attain desirable theoretical properties" and not "I'm failing to understand something". Anyway, I found a book on this stuff, which builds up to "System Fω with subtyping", which is, um, where I want to start in terms of this project.

So, I figure I'll work through that book; while it's possible that I can simply muddle through things with my current level of knowledge, getting a firmer foundation should only be helpful, aside from the fact that it may take more time than just charging ahead and getting lucky.

As such, I'm going to try to switch gears to other projects, at least as far as blogging goes, and read up on this in the background.

...

And then I went and got distracted by another project that I'm nowhere near ready to blog about. Oops.

Going to get to bed now. Maybe read the intro to that book.

Good night.

Coding 2024-12-28

2024-12-28T05:00:00-05:00

All right, with a bit of thought, I understand that the different papers I was reading were using the word "kind" in compatible fashions. The gamble I have to take is to create several sorts of "field" and "extension" kinds, as well as a "variant" kind, and maybe some other stuff. From there, the key is to make sure that I have type inference rules that are compatible with the new kinds.

I'm reading up on type inference techniques that work with subtyping, which is going to be a must to deal with the row types that all of that is meant to introduce. The complication I have to deal with right now is, I kind of want three things that could reasonably be described as row types, and there are a bunch of questions I need to answer:

What are the associated kinds?
What are the corresponding type inference rules?
Does this break anything at a theoretical level?
Practically speaking, how do I translate this to code?
I want to implement these things one at a time; how do I lay out the code so that grafting on more kinds like this isn't a complete pain?

Oh yeah, also: I guess I'm going to need more types (kinds?) at some point for FFI purposes, because I'm planning to have a level of flexibility for internal types that doesn't sensibly apply to external types.

Anyway, it's best if I plan before further prototyping, best if I read more before planning, best if I sleep before reading.

Good night.

Coding 2024-12-27

2024-12-27T05:00:00-05:00

Okay, let's see. As a result of manually translating the diagrams in these papers into code, I'm getting some idea of how both the notation and the concepts work.

Things I still need to figure out:

How should the typing rules be expressed in code?
Do these papers written decades apart by different people mean the same thing when they say "kind"? If not, what is the proper way to reconcile the differences?

I'll see where this ends up, but for now, I need to wind down.

Good night.

Coding 2024-12-26

2024-12-26T05:00:00-05:00

Okay, I've got some idea of the AST for System Fω, and a somewhat vaguer idea of how I'm going to try to extend it, and a still fuzzier idea of how I'm going to try to turn this stuff into executable code.

I should have more time to mess around with this tomorrow, so then I can either try to shore up my theoretical understanding, or just throw some code together and probably end up being all "it broke" when I ask for help.

For now, I need to get ready for bed.

Good night.

Coding 2024-12-25

2024-12-25T05:00:00-05:00

Okay, I got kind of distracted thinking about the surface syntax today, so I didn't make much progress in the direction that I wanted. To try and get something done today, I'm going to go through the target compilation flow I think I'm going for.

Write source code in Impliciula.
Parse to high-level AST.
Desugar AST.
Generate type constraints and resolve all constraints with z3. (Is this going to be multi-pass or otherwise incremental?) (Wait, maybe there's a SAT-solver in Rust that I can just use directly?)
Possibly do a few more AST transformations, until we get to something with a mostly limited vocabulary.
Do What It Takes to convert that to LLVM IR.

Now, even given the amount of slop and slack in all of that, I feel like the right way to proceed is to make sure that the end steps can even happen. So, the proper stuff to focus on is extending System Fω until it has the capabilities I want (several new kinds of type, as well as a few new kinds of expression), somehow taking care to avoid breaking the desirable properties of System Fω. From there, I can construct modules, and higher-level syntactic sugar.

As to how to extend System Fω, my plan right now is to write down my goals so I can refer to them later, and then, um, teach myself the relevant aspects of CS so I sort of know what I'm doing. We'll see how this goes. Later.

Good night.

Weekly Roundup 2024-12-24

2024-12-24T05:00:00-05:00

Wednesday: I changed some names in MOTR. It's somewhat awkward, but better than it was before.
Thursday: Not much, but I do have a source to use for some of my writing, which is something.
Friday: Nothing much.
Saturday: I found something that could have a huge effect on Impliciula's design.
Sunday: I became more confident that the papers I found are important.
Monday: I tried to nail down the new direction for Impliciula, and honestly it looks like several research projects in a row, but I'll try!

Next week, I'm going to try to fill in my very high-level plan for Impliciula, and also, uh, learn to write a compiler.

Coding 2024-12-23

2024-12-23T05:00:00-05:00

Okay, let's see what we've got here.

1ML defines a core language without type inference, which can be translated to System Fω. The paper I'm reading gives the typing rules for System Fω, so I'm currently figuring that the idea is to translate down to System Fω and type check it in that form?

Given that, my plan of attack looks something like this:

Implement 1ML without effects.
Extend it with aspects of F-ing modules that the 1ML paper leaves as an exercise for the reader.
See what goes wrong if I attempt to implement row typing in the core language. (Reading ahead, I see that this was considered, and specific pain points identified.)
Extend row typing to the point where it can handle what I have in mind for effects.
Implement a basic form of tail-resumptive effects.
Add further capabilities until I'm satisfied.
Somewhere in there, try to handle predicates some way.

One thing that's fairly obvious to me, looking at the papers, is that I'll need to have some kind of trace or source map associated with the different translations, because an error in terms of System Fω would be entirely incomprehensible.

In any case, reading over the papers, I'm cautiously optimistic about integrating row types, but it may end up being the case that I end up hacking on System Fω instead of the 1ML core.

For now, though, I need to wind down.

Good night.

Diary 2024-12-22

2024-12-22T05:00:00-05:00

Okay, I've looked enough at the papers for 1ML and F-ing modules to be confident that there's something here for Impliciula, but I'm not quite ready to commit to anything yet.

Aside from that, travel, and that means that I'm kind of fried. It is currently way too late, and I want to stop looking at bright things.

Good night.

Diary 2024-12-21

2024-12-21T05:00:00-05:00

Okay, it's time to relax. Over the next few days, I'm going to try to scale things up with MOTR and some of my writing, but right now, I'm just going to take things easy.

Also, I just ran across 1ML, and that could be a huge can of worms when it comes to Impliciula's design. Maybe I'll be thinking about that soon, but not now.

Good night.

Diary 2024-12-20

2024-12-20T05:00:00-05:00

Didn't manage time well today, so I have nothing to discuss. We'll see what shakes out tomorrow, and over the next few days.

Good night.

Diary 2024-12-19

2024-12-19T05:00:00-05:00

There was a lot going on today, and as a result, I didn't get too much done.

I did come across a crucial source for some of my non-fiction writing, so that's good; I'm going to have challenges properly integrating its messages, though.

Right now, though, it's vital that I get some rest.

Good night.

Coding 2024-12-18

2024-12-18T05:00:00-05:00

Well, I started renaming things, and took a while to stop, and shockingly, it all merged totally cleanly to the future topic. I think at this point, the next step, at least in the CLI wrappers, is to take some time away from them, and come back in a few days with fresh eyes to see what looks not-quite-right now.

While I'm looking elsewhere, I notice some code duplication that can be trivially removed, so I do so, and ratchet up the coverage a little bit more in the process.

I'm not sure what to work on next, and I'm getting too tired to work it out, so let's wrap things up here.

Good night.

Weekly Roundup 2024-12-17

2024-12-17T05:00:00-05:00

Wednesday: I made a bit of progress on MOTR.
Thursday: I managed a tiny coverage improvement.
Friday: I made an improvement to typing coverage, and noticed a problem with the interfaces as a result.
Saturday: I started to come up with a more detailed mental model for parts of the high-level abstraction in MOTR than "the thing" and "the thing that the other thing goes in".
Sunday: I discovered a wrinkle in every version of the relevant model, and mostly just chose to ignore it for now.
Monday: I tried to write up these ideas a little more clearly, and come up with a concrete improvement I can make based on them.

Next week, I'll try putting that into action.

Coding 2024-12-16

2024-12-16T05:00:00-05:00

Well, I was focused on other things today, so I'm going to try and get some stuff written quickly, then wind down.

The last few days, I've been toying with the concept of foregrounding the interpretation of high-level command abstractions as an inhomogeneous linked list. These linkages are formed via two methods on the command head/cap class, which attach the instance to a method-specific type of tail, and the result, depending on whether the methods are called on a head or a cap, is a tail or a completed list, respectively.

And as I type that up, I suddenly recognize one of the problems with the current abstraction. I was thinking yesterday about writing a new tail class, but if the way to attach a tail is to call a method on the head/cap, then every new tail class needs a new method on the head/cap class. This method should instead be part of the command tail interface.

To help preserve the order of data in the code, there could perhaps be a convenience method on the head/cap class, although the typing may be somewhat confusing... The problem with that idea is that the different kinds of tail have different effects on the typing of the head/cap when it's prepended, and I don't yet see a way to address that.

Given that it looks like I won't be able to change the shape of the interface as much as I hoped, it's best to focus on the names for now. The shared prefix for the methods to attach to a tail is with_, and my current feeling is that prepend_to_ is a lot better. The names can then be something like prepend_to_script_environment and prepend_to_module_call.

I'll have to sleep on that, and come back to it in a few days.

For now, winding down.

Good night.

Coding 2024-12-15

2024-12-15T05:00:00-05:00

I was mostly relaxing today, but I did take some time to consider the ideas I had last night. I'm not sure if the names that I have in mind are quite right. They should be an improvement on the basis of distinguishing types with different behavior, but I don't know if I'm using the right metaphor yet.

Basically, by viewing commands as something like a linked list of arguments, we can build up a "tail" by adding some "heads" to it, and then adding a "cap" to "complete" it.

I suppose figuring out whether this choice of terminology is "right" hinges on figuring out the target audience. Unfortunately, of the four interest groups I've identified for working with MOTR (developers of unrelated repositories, maintainers of unrelated repositories, developers of plugins, and core developers), fully three of them care at least somewhat about understanding whatever terminology I come up with.

On the one hand, I can always change this later, but on the other hand, it would be nice to get it right now.

I also have to consider the possibility that this "cap" concept may not even be correct. After all, some command-line invocations go some-cmd arg1 arg2 -- some-other-cmd-entirely arg3 arg4. In other words, it can be a property of some caps and heads that they can go on top of what should be a cap.

...

I can't go back to TF2.

Boo!

Corny jokes aside, I'm getting the feeling that I can awkwardly patch I mean extend this by writing an adaptor class that converts a completed list to a tail. The main barrier to doing so is the dire quality of the current documentation in this area. And the documentation is so bad because I didn't know what to call anything...

Okay, I've got to at least try the head/tail/cap/complete writeup and see if anything comes of it.

For now, though, I need to wrap up.

Good night.

Coding 2024-12-14

2024-12-14T05:00:00-05:00

Okay, I've been renaming stuff in MOTR's upper layers, trying to gradually reduce the technical debt I incurred by pushing through the unintuitive stuff. At this point, I'm getting closer to figuring out what to call ParametricCommand and ParametricCommandMeta. Basically, a given CLI wrapper has to contain an object that is one of two types, but actually there are kind of four of them, and I'm trying to figure out a naming scheme. My ideas here are kind of inching towards Apps Hungarian notation, so we'll see how this all goes.

Basically, the way I'm thinking about this is kind of like a 1-dimensional jigsaw puzzle where there are only two types of edge. To actually create a Matrix that will put the desired Facts into the Compendium, we need a ParametricCommand[Script, Any, Any, Any], which is like a solved puzzle. The unsolved pieces are ParametricCommandMeta[Script, Any, Any, Any] (solved on the left, not on the right), ParametricCommand[Module, Any, Any, Any] (solved on the right, not on the left), and ParametricCommandMeta[Module, Any, Any, Any] (unsolved on both sides). (One minor complication is that currently the "puzzle" has to be "solved" right-to-left, but it's probably fine.) Furthermore, there is another type that is basically a puzzle piece solved on the right, not on the left. This is needed because it's not possible to construct a ParametricCommand without a ParametricCommandMeta, so there needs to be some way to "solve the right side" as part of converting.

I wonder if I could get better names by considering all of this from the angle of "I mean, it's kind of a linked list". Like, "command head" and "command tail". I think I can get somewhere with this idea, but I also think I'm done for tonight.

Good night.

Coding 2024-12-13

2024-12-13T05:00:00-05:00

As I look at MOTR's code with fresh eyes, I discover a gap in the typing coverage. This first manifested in some light boilerplate code that I realized wasn't accomplishing anything. And it wasn't accomplishing anything because there's no connection between where it's defined, and where its definition is supposed to propagate to. I'm going to try to fix this, just to see what problems it causes as a result...

Well, the code looks a little weirder, but things should be actually hooked up now, probably...

Well, I managed to inline enough code that my code coverage fell by 0.01%. My hope is that the modules are a little more focused now.

Looking at this code with fresh eyes, one issue that I'm seeing is that different modules are using the same name for a value with different types, but the fact that the type is different may not be immediately obvious. I'm going to have to redo these names.

For now, I'm incredibly tired, so I'll be getting to bed.

Good night.

Coding 2024-12-12

2024-12-12T05:00:00-05:00

So. I made a few more small tweaks to MOTR's structure, and then I decided to get back to increasing coverage. Unfortunately, some of the coverage misses are related to code for the CLI wrapper layer, which I believe really needs a rewrite still, and some of them are just... really quite confusing. Like I'm going to have to devote a lot of time just to figuring out when a particular method should even be called.

Although, I did just manage to squeeze out a tiny bit of coverage. Progress.

I shouldn't make any more progress tonight though, except for progress towards bed.

Good night.

Coding 2024-12-11

2024-12-11T05:00:00-05:00

I'm sort of jumping between projects, but the quantifiable achievement of today is that I redid some of the package layout, and got it working with the latest Mypy. (The trick was to stop using a particular third-party plugin that it turns out became unnecessary at some point.)

This was pretty subtle overall, but I'm hoping it kind of compounds into getting everything to make more sense.

I've got some ideas about other stuff, such as Impliciula, but they're not quite baked enough yet.

Anyway, it's late and I should get to bed.

Good night.

Weekly Roundup 2024-12-10

2024-12-10T05:00:00-05:00

Wednesday: I thought about MOTR and conlanging.
Thursday: Conlanging, and using my laptop for way longer than I should have.
Friday: I started talking about the language concept, Divine. It's for pretentious jerk fantasy races.
Saturday: I tried to figure out the implications that Divine's sentence structure has for parts of speech.
Sunday: I had nothing really to say.
Monday: I had nothing really to say, but also a headache.

Next week, I'm going to try to make some progress of some kind on something.

Diary 2024-12-09

2024-12-09T05:00:00-05:00

Good news: I've taken inventory of my current projects, and I should be able to methodically make progress on them.

Bad news: I did this during the car ride back, and I'm still feeling motion-sick as a result.

I'm going to post this, put on some music, and read.

Good night.

Diary 2024-12-08

2024-12-08T05:00:00-05:00

I made some progress on some projects today, but nothing that I'm ready to show off. I'm very tired and I should get to bed. Oh well.

Good night.

Conlanging 2024-12-07

2024-12-07T05:00:00-05:00

Hm. Another day, another weird mental state. Let's just work through some sentences quickly and see what difficulties they present.

The lookout sees the soldiers as clearly as the eagle sees the soldiers.
The lookout sees the soldiers as clearly as an eagle would see the soldiers.
The lookout sees the soldiers as clearly as the eagle sees its prey.
The lookout sees the soldiers as clearly as an eagle would see its prey.
The lookout sees the soldiers as clearly as the elephant hears the soldiers.
The lookout sees the soldiers as clearly as an elephant would hear the soldiers.
The lookout sees the soldiers as clearly as the elephant hears the rockslide.
The lookout sees the soldiers as clearly as an elephant would hear a rockslide.

I'm mostly interested in the result of variation along particular axes, but I figured I'd fill in all of the possible combinations.

The idea I'm looking for is to move parts of verb phrases to particular locations. The full "deep" phrases are:

"sees the soldiers clearly"
"would see the soldiers clearly"
"sees the eagle's prey clearly"
"would see the eagle's prey clearly"
"hears the soldiers clearly"
"would hear the soldiers clearly"
"hears the rockslide clearly"
"would hear a rockslide clearly"

... and those don't actually present the problems I was worried about.

The lookout sees the soldiers as clearly as the clouds disperse quickly.
The lookout would see the soldiers as clearly as the clouds would disperse quickly.
The lookout sees the soldiers as clearly as the eagle sees its prey hungrily.

If I were editing this, those sentences would be less awkward, but the awkwardness shouldn't interact with the grammatical questions I'm trying to deal with. Those sentences deal with the idea of dividing the verb phrase parts into, like, TAM, action, manner. The thing I'm trying to figure out is if I want them to all be separate constituents, or not. Part of the issue is that I want to make all of the basic words be verbs.

...

I just realized that doing this with transitive verbs made everything more confusing for me.

The man runs faster than the dog runs.
The man runs faster than the dog eats.
The man runs faster than the dog runs gracefully.

Some of these sentences seem nonsensical or awkward, but the Divines would explain that this is simply the result of our quaintly limited perspective.

The more I think about this, the more I think I'm going to need to make use of anaphora, but in a way, that just makes it better, because now there are a bunch of weird special-purpose words.

Also a lot of weird reflexives, but I don't have the energy to work that out tonight.

Good night.

Conlanging 2024-12-06

2024-12-06T05:00:00-05:00

Okay, so, I was thinking about other projects today, but I can explain where I'm at with this conlang concept.

Lately, my favored entry point for conlang development has been to come up with a grammatical feature or construct I want to mess with. (If someone else would take responsibility for the bulk of the worldbuilding in a project, I might be able to make progress on the conlang aspects, but given that my major worldbuilding project right now is currently hovering at "can I make eukaryotes exist in this setting?"...) Anyway, my inspiration here was the concept of "what if the snooty ethereal beings were so snooty that their language considered 'we are good' a sentence fragment, and the grammatical version is something like 'we are better than you'?" Basically, all sentences have to be explicitly a comparative construction, with the stated reason that they're aware of the broader context for everything, and speaking in absolute terms is philosophically incoherent. Why, someone speaking a less refined language might say that a room is cold, but that implicitly privileges their own perspective, and such an utterance would be worthless to, say, a polar bear.

Right now, I'm ignoring the question of phonology, roots, etc, and just trying to sketch syntax stuff. I've got a bunch of ideas about movement or whatever, that may not actually make sense. I'm going to have to work through a few examples, and see if I can make it work, or if I need a bunch of anaphora.

Anyway, it's late and I should get ready for bed. Hopefully I'll be able to get some of this work done tomorrow.

Good night.

Diary 2024-12-05

2024-12-05T05:00:00-05:00

I had more symptoms of something today, and did get a bit of rest. But I also did a small amount of conlang work. I think I've got this one to a point where it makes sense to blog about what I'm trying to do with it, but I'm not ready to talk about anything in detail right now.

Right now, sleep.

Good night.

Diary 2024-12-04

2024-12-04T05:00:00-05:00

I was fighting off something migraine-esque today, so I probably should have actually made this post even shorter. Oh well.

I'm still pondering stuff about MOTR, but I haven't yet convinced myself to touch any of the actual code yet.

Aside from that, I'm reading stuff that's inspiring me to do stuff with one of my conlang ideas, so I'm trying to get something done there, but I don't feel like I've got anything to write about yet.

Good night.

Weekly Roundup 2024-12-03

2024-12-03T05:00:00-05:00

Wednesday: I tried to understand what the COCHIS paper was saying about Scala.
Thursday: I decided to move on from that specific section for now.
Friday: I determined that I don't understand System F well enough to understand COCHIS.
Saturday: I decided to muddle through, and to try to adapt COCHIS to the ML-like framework I want to work in.
Sunday: OCaml is complicated.
Monday: I decided to take a break from Impliciula, but came up with a general plan for how to proceed when I feel I'm ready.

Next week, we'll see. I'm working on revisiting MOTR, and I'm also poking at some other projects that I'd put down for a while.

Diary 2024-12-02

2024-12-02T05:00:00-05:00

Traveled today, after a rough night's sleep.

I've been thinking some about how I want to approach Impliciula development. On reflection, it seems like it makes the most sense to decide on a subset of designs in the ML lineage, and then work on implementing that, without any of the experimental features or quirky syntax ideas.

Anyway, I spaced out and it's late. Time to get ready for bed.

Good night.

Coding 2024-12-01

2024-12-01T05:00:00-05:00

Ugh.

No real progress on Impliciula. There's just too much of OCaml for me to casually take in.

I'm going to have to find something else to focus on, and some other way to get the understanding I need for this.

I'm trying to pick up my other projects for now; we'll see how that works out.

Good night.

Coding 2024-11-30

2024-11-30T05:00:00-05:00

I did not have the best of luck understanding System F vs COCHIS vs OCaml, so I'm just going to take some quick notes and get to bed.

The COCHIS formalism presents things in terms of "deriving an implicit value (of some type) from some type", while OCaml's modular implicits were more like "deriving an implicit module (conforming to some signature) from some type". My gut feeling is that this won't produce any major issues to adapting COCHIS, but I could be wrong.

Also, I think I finally managed to translate the problematic Scala code into an OCaml-style system, and the resulting code is deeply un-idiomatic, but also I'm pretty sure it gets disallowed by aspects of COCHIS that were more chosen for ease of implementation than to guarantee stability. So, ultimately, that's good for now, because I don't think I have an objection to that part of COCHIS.

Back to the difference between deriving types and modules, that does mean that I think it should be possible to have an implicit function where there's an essentially arbitrary relationship between input types and output types, but for one thing, I could be wrong, and for another thing, I guess that would be of limited use, but maybe I'm missing something.

Anyway, it's late, and I should have wound down a while ago, but I'll settle for winding down now.

Good night.

Coding 2024-11-29

2024-11-29T05:00:00-05:00

Hm. I'm reading over the paper, and it's talking about presenting COCHIS in terms of System F. I should have experience with System F from working with OCaml and Haskell, but I was thinking I'd try to approach it from a stripped-down theoretical perspective, but I didn't make too much progress; I now have a general hand-wavy understanding of simply typed lambda calculus and System F, but I wouldn't expect too much of myself just yet. I'm going to give the paper another shot now; plenty of time before posting.

...

All right, I've skimmed over a bunch of the paper, and I'm going to try putting together prototypes based on its description of COCHIS.

...

Okay, what I've figured out from this is that I need a better handle on System F, particularly how to handle "environments", because that's something I didn't have in the simpler lambda calculus prototypes I knocked together today.

As such, my plan now is to get a handle on System F implementations and write one myself to hack on and extend in accordance with the COCHIS paper's description.

For now, I need to sleep off everything that happened today, intended and not.

Good night.

Coding 2024-11-28

2024-11-28T05:00:00-05:00

Okay, I spent some more thought on the question of "what if there's a generic function and a specific function", and on reflection, I think I do need to handle it somehow. Encountering the behavior seems like it involves un-idiomatic code, but I can't wriggle out of this by being all like "well, you shouldn't write that".

The question I need to ponder (and figure out how to test in OCaml or read the documentation on) is, "does a module containing a generic function satisfy a module signature containing an instantiation of that generic function type?"

...

Actually, that's a good question, but looking at specifically the scenario on page 12 of the COCHIS paper, I feel like there's something Scala is allowing that the design I'm groping towards does not, but I can't really tell for sure. I'll have to just accept that this is apparently a problem in Scala, and read the rest of the paper.

I feel like wrapping up this post early-ish and going through as much of the paper as I feel like after.

Good night.

Coding 2024-11-27

2024-11-27T05:00:00-05:00

So, these last couple days, I was distracted by something I don't think I'll ever explain here (except that I was wrong to doubt myself), but that should be over with now.

Anyway, let's try to get through a bit more of the COCHIS paper before I get ready for bed. As I said before, I got to the Scala section, and I'm having a bit of trouble due to having no Scala experience. I can contrast what I'm seeing so far with how I want to do things in Impliciula, but I'm not sure if there's any point to that; I haven't read far enough to get a sense of what parts of Scala's design they want to highlight. Let's see if I notice any key points in the next few minutes...

"Scala implicits do allow for a wider range of applications than [Haskell] type classes." It then gives an example of this.

Following that, it gives an example of an instance of instability in Scala. I should take the time (which I don't have right now), to carefully study and translate the example, because I don't think I have any reason to expect my current design ideas to just happen to avoid the issue. It could happen, but it would be complete luck on my part.

From a brief look at the example, my intuition is that a naive translation would fail to compile, but it may be possible to "salvage" the instability in some way.

Anyway, it's late; I'd better pack up, etc.

Good night.

Weekly Roundup 2024-11-26

2024-11-26T05:00:00-05:00

Wednesday: I started outlining a book or something, and had some thoughts about tachyons.
Thursday: I started writing a solver for an online puzzle.
Friday: I got the solver written; the output is pretty janky, but it worked.
Saturday: Out of it.
Sunday: I was still kind of out of it, so I made plans instead of rushing.
Monday: I gave a shot at those plans, sketching out how the current idea of Impliciula handles some stuff from the COCHIS paper.

Next week, we'll see how I feel.

Coding 2024-11-25

2024-11-25T05:00:00-05:00

All right, I'm looking over the COCHIS paper, and I want to try just translating the examples to my sketch syntax for Impliciula and see where things go wrong.

The first sample to look at deals with ordering, and depends on the less-than-or-equal-to function. We might translate this arrangement something like:

let module type Ord be signature
  type t
  {<=} (t, t -> bool)

let {<=}
  with implicit module O(Ord where t is 'a)
be ('a, 'a -> bool)
  O.{<=}


let implicit module IntOrd be
  let type t be int
  let {<=} be (int, int -> bool) Int.{<=}

let implicit module CharOrd be
  let type t be char
  let {<=} be (char, char -> bool) Char.{<=}

let implicit module PairOrd
  with implicit module O1(Ord) and implicit module O2(Ord)
be
  let type t be O1.t * O2.t
  iff v0 (t) <= v1 (t) then
    v0[0] <= v1[0] and {not {v1[0] <= v0[0]} or {v0[1] <= v1[1]}}

let sorted: l (list-of: 'a)
  with implicit module O(Ord where t is 'a)
be (list-of: 'a)
  ...

Among other things, I haven't really nailed down the tuple syntax. Regardless...

I'd say a key difference between that sample and the bits of the COCHIS paper that I'm adapting is that I'm using modules, so they're not explicitly parameterized by the type, and instead have to match it with where-clauses and attributes. This might or might not totally break COCHIS, we'll see.

Reading ahead, we see that, at least in Haskell, a blanket implementation of a typeclass allows leaving the typeclass requirement off of a function definition. (I think. I'm not totally sure I'm reading this right.) That sounds like something I don't want to do in Impliciula.

Anyway, I got to the Scala section and my eyes glazed over a little. I'm going to assume that's a "me" problem, and I should start winding down a little early.

Good night.

Diary 2024-11-24

2024-11-24T05:00:00-05:00

Hm. I was just kind of fried today, so if I'm going to have anything to talk about, I'm going to figure it out and write it up right now.

Nothing coming to mind for tonight, so I'm just going to try to get my thoughts in order about stuff I can work on in the future. I recently read a blog post about modular implicits that mentioned a few things about them that I hadn't realized; like that the reason they haven't achieved much penetration in the past decade is that the precise details of a "good" implementation haven't been nailed down yet. The post also mentioned a paper on a system called COCHIS that could address some implementation issues.

So, what I need to figure out for Impliciula is whether it's possible to put the following ideas together:

COCHIS semantics
Within an ML module framework
Possibly with some kind of "default" setup that is subject to similar constraints as the trait system in Rust.

Something like, you either have to own the signature or every public type in a module to make it a default implicit? No idea right now whether that works and even makes sense.

Anyway, I really should stop staring at this screen.

Good night.

Diary 2024-11-23

2024-11-23T05:00:00-05:00

Oh boy, I let things go way too late. No point trying to write anything more substantial. Have to sleep.

Good night.

Coding 2024-11-22

2024-11-22T05:00:00-05:00

About a hundred lines of Python later, I've got a solver working. We'll see if this gets me to put Domino Fit down.

It's late and I don't have anything else in mind to talk about. Maybe something new tomorrow.

Good night.

Coding 2024-11-21

2024-11-21T05:00:00-05:00

Recently, I've been playing a daily puzzle game called Domino Fit. It's really interesting, but after I discovered some ways of getting bragging rights in the score screen, I've been thinking about it way too hard. As such, I'm trying to code up a solver so I can, um, exorcise my urge to play it.

I don't have anything to show off yet; I'm still working on encoding the state of everything into z3. I'm working inspired from How to Solve the Sudoku Puzzle with programming, which I swear is a real post, except I'm using the Python bindings to z3 instead of generating DIMACS.

I should have something that works in a day or two, but at the moment, I once again really need to get ready for bed.

Good night.

Diary 2024-11-20

2024-11-20T05:00:00-05:00

Okay, I'm working on outlining for that writing. This is going to be some kind of book, or zine, I don't really have an idea of the length. I guess it's a "manifesto"? We'll see.

Anyway, aside from that, I realized today that thinking about energy-momentum vectors in a principled fashion gives relatively straightforward descriptions of how tachyons would work, so naturally I tried to figure out what it would imply. Various issues arose, such as "it looks like a particle following a spacelike trajectory should be able to bounce off of events in time", and "hey, wait, multiple particles individually following spacelike trajectories can have a combined energy-momentum vector that points in a timelike direction, what the heck?" and "I don't know what to call the minimum magnitude of a tachyon's momentum, which it attains at infinite speed, (which also gives it zero total energy)".

Basically, it turns out that there are problems way more interesting than blindly plugging in for the Lorentz and going "oops, looks like the mass is imaginary". It seems to me that different trajectory types have different associated quantities: rest mass, energy, and, um, the fundamental quantity for tachyons seems to be something that Wikipedia doesn't list a derived unit for. Is there a single-part name for the "Watt-meter"?

Anyway, I was messing around and I'm up too late.

Good night.

Weekly Roundup 2024-11-19

2024-11-19T05:00:00-05:00

Wednesday: I was learning the Shaw alphabet, even though I'm still not sure why.
Thursday: Pain.
Friday: Relaxation.
Saturday: I made some efforts to apply what I've learned with the Shaw alphabet to... something.
Sunday: We traveled, and I didn't have anything to say.
Monday: I came up with something to work on, but I don't want to talk about it yet.

Next week, I've got a few things I can look at. (Turns out I didn't really want to make fun of Terrence Howard as much as I thought I did.)

Diary 2024-11-18

2024-11-18T05:00:00-05:00

Traveled back. I have an idea for stuff to write that's likely going to be challenging, but I think it has the potential to be really helpful, um, whenever I finish it. Which could take a while. Whatever; the need for it isn't going to vanish over time.

I don't want to describe it right now, because I've just barely gotten started and I want to keep up my motivation; there's also a good chance that the concept ends up very different from what I'm currently envisioning. There are some ideas I have that I know for sure are going to be hard to pull off.

For now, I should get ready for bed. Maybe get some quick Shaw alphabet practice in.

Good night.

Diary 2024-11-17

2024-11-17T05:00:00-05:00

Weird day today. Some travel, so I'm writing this somewhat late. I'm messing around a little with the Clifford universe worldbuilding, but nothing really interesting yet.

I was trying to think of something else to say, but I don't have anything else in mind apparently, so I'm going to wrap things up.

Good night.

Diary 2024-11-16

2024-11-16T05:00:00-05:00

Still taking things easy. I'm still looking for direction with my projects. Learning the Shaw alphabet has inspired me to attempt similar-ish things for other languages, with the obvious caveat that no native speaker of these languages has asked me for such a thing.

Maybe I can repurpose this draft to be for another language, or try doing this with a conlang.

Anyway, I'm sleepy and I don't have anything I should be doing currently. Going to wrap things up.

Good night.

Diary 2024-11-15

2024-11-15T05:00:00-05:00

Well, I think that worked. I don't think I'll know for sure until morning.

I'm... not sure I'll do much of anything until morning. I guess that was sort of the idea?

Anyway, I'm going to go space out.

Good night.

Diary 2024-11-14

2024-11-14T05:00:00-05:00

Uuuuuuuuuuuuuuuuuuuuuuuuuuugh...

I'm going to get blasted tomorrow and see if that fixes this.

For now, I don't really have anything to do or talk about.

Good night.

Diary 2024-11-13

2024-11-13T05:00:00-05:00

I'm still fighting off a migraine, which is doing my ability to focus no favors whatsoever. Unfortunately, the intellectual awareness that it's good sense to rest and not push myself isn't helping me deal with the fact that I want to be writing, and coding, and maybe learning other things, and it's not helping me be okay with the fact that the thing I'm doing right now, I don't know why I'm doing it.

𐑤𐑲𐑒, 𐑢𐑫𐑛 𐑧𐑯𐑦𐑢𐑳𐑯 𐑚𐑰 𐑦𐑯𐑑𐑼𐑧𐑕𐑑𐑩𐑛 𐑦𐑯 𐑮𐑰𐑛𐑦𐑙 𐑚𐑤𐑪𐑜 𐑐𐑴𐑕𐑑𐑕 𐑦𐑯 𐑞 ·𐑖𐑷 𐑨𐑤𐑓𐑩𐑚𐑧𐑑?

I've got some general idea that I could use this stuff in graphic design or something; the pure-English version of, like "this tattoo says 'wisdom'" or whatever. I don't know. And I'm probably not going to figure it out while my neck hurts this much, but accepting my limits here feels like giving up.

At the same time, what else am I going to do right now? I really should just finish this post and try to fix my neck.

Good night.

Weekly Roundup 2024-11-12

2024-11-12T05:00:00-05:00

Wednesday: I prepared to read and mock One Times One Equals Two by Terrence Howard.
Thursday: I used that as a jumping-off point to look at The Universal One by Walter Russell, which is where Terrence Howard gets some of his random diagrams from.
Friday: I tried to analyze the differences between Terrence Howard's interpretation of Walter Russell and mine, but I got sidetracked by the fundamental, overriding, pervasive flaw in Terrence Howard's writing, which is that it's just really obviously stupid.
Saturday: I pointed out a bunch of things that Terrence Howard doesn't understand nearly as well as he thinks he does (when I was listing them, I forgot Latin), and then got excited because he mentioned Norman Wildberger, which is a good enough excuse to read an "alternative math" book by someone who's at least competent.
Sunday: The wheel of hyperfixation unexpectedly spun, and landed on "learn the Shaw alphabet", so, um, okay.
Monday: Handwriting gave me some trouble, but it was mostly the pen's fault.

Next week, I'm either going to figure out how to keep on writing blog posts from trying to learn Shavian, or I'm going to find something to say about Divine Proportions.

Diary 2024-11-11

2024-11-11T05:00:00-05:00

I've got some tech for handling the Shavian alphabet on my various electronics, and right now I'm focusing on handwriting it. Not totally sure where I'm going with any of this, but I'm going somewhere, presumably.

I did have a bit of trouble with the new pen I was using failing to put down ink. I... may have called this "𐑒𐑢𐑽𐑓𐑴𐑚𐑾" as a joke. I wonder how the fonts I'm using will cope with Shavian text; I can say for sure that whatever my terminal is doing, it's pretty disappointing. Like, why are the letters getting the thinnest possible stroke?

Oh well, hopefully it works out. We'll see.

Anyway, I'm tired and I want to wrap up.

Good night.

Diary 2024-11-10

2024-11-10T05:00:00-05:00

Okay, I didn't have the focus today to read more of Terrence Howard's incoherent nonsense, or to argue with Norman Wildberger's, um, coherent statements. See, the nice thing about Norman Wildberger is that I can look at specific points and decide whether I agree with them, rather than just gesturing vaguely at word salad.

Anyway, for whatever reason, what I felt like looking at today was the Shavian alphabet. Not sure where I'm going with this, but I'll mess around with it for a bit. I should dig out my sketchbook and start messing around. Anyway, for now I should get ready for bed.

Good night.

Terryology 2024-11-09

2024-11-09T05:00:00-05:00

Oh boy. I've gotten to page 39, so Terrence is mixing it up a little. Not content with being wrong about arithmetic, physics, chemistry, and probably other things I've forgotten, he branches out into saying incoherent things about English grammar. This sort of ties in with a general theme in his work, where he tries to define the behavior of reality in terms of (his understanding of) the English language. Like, "multiplication, which by definition means, 'To make greater.'" on page 24. On page 39, he says that "In long form writing it is expressed as follows 'One time(s) ␍ One.'" I don't think I've ever seen that, and then he tries to argue that the (optional according to him?) s makes it a plural. There are actually so many different things wrong with this that I'm having trouble deciding how many there are. One is that the word being used for the multiplication operation doesn't have anything that could be a plural marker in other languages, like Spanish "por". This segues into another related issue, which is that we can just as easily say "one by one" in English. However, it's important to note that mathematical terminology can sometimes be a fraught area when it comes to grammar. In English, numbers go before adjectives, but mathematical expressions are used as arguments to verbs, so they ought to be nouns. And the usage of "by" and "divided by" seems to indicate that mathematical operations are prepositions, but we understand indicating addition with "and", and when it comes to indicating more advanced operations, it's often the case that they come after the number, like "squared" or "factorial". These could be sort for "the square of" and "the factorial of", but the abbreviation scheme isn't consistent: consider how "the square root of" becomes "root" before the number. To say nothing of "sine squared", in which "squared" now comes before the number, or is it after "sine"? Despite the weird number of things going on, this all works well enough, as a means of communication, and not some kind of magical language to bind the nature of reality.

I mean, ultimately, nobody is reading this looking for a proper debunking, because I'm pretty sure nobody else really believes what Terrence is saying.

What about the editor?

This book had an editor?

I mean, it says it had an editor, and last we checked, that person had a website with computer generated images like the ones in the book...

Truly, a deeper mystery than any question Terrence raised on purpose.

But enough snark, let's see if I can find anything interesting.

Again, if we are discussing Light and the unquestionable constant of the speed of light, a measurement that we are told is unchanging. Light is believed to travel faster than most things within our 3-D Universe. Due to the seemingly instantaneous transference of information across the electrified Universe as observed in Birkeland Currents, we know that electricity is not limited to the speed of Light, it is much faster. Yet, Light a radiative product of a magnetic interaction isn’t doing so bad coming in a close second. Remember, Radiant Light moves in the opposite direction of Electricity and Light’s centrifugal expansion is at a higher ratio than the centripetal contraction of Electricity, which leaves behind the effects of gravity, as a by product!

This was kicked off by him recounting one of his elementary school teacher's presumably increasingly desperate attempts to get him to do multiplication properly. (I mean, the way he tells it, I can't figure out where she was going with it.) There's a lot there, and I'm fairly confident that none of it came into play at the time. Let's try to salvage some points of legitimate interest from all of this gawking:

I kind of glazed over a bunch of pages where Terrence was talking about, like, Einstein, and Superman, and Pythagoras, and stuff. Does he explain anywhere what he means by "Radiant Light"?
What are Birkeland Currents? Who claims that they exhibit ftl effects?

Let's see:

That is, in fact, the only time he uses the word "Radiant" in One Times One Equals Two, so I'm just going to ignore it for now.
Birkeland currents are a geomagnetic phenomenon that, among other things, cause the aurorae. It has been hypothesized that a similar phenomenon could be responsible for the periodic radiation patterns of particular stars, through the creation of waves that "move faster than light" without carrying information or particles.

Let me break out of the list to try to explain. Imagine you have a very powerful laser pointer, and you're standing at the center of a cylindrical room ten light-minutes in radius. (Please ignore the material engineering issues inherent in this hypothetical.) You turn on the laser pointer, and set it horizontally. Twenty minutes pass, and the dot appears. (It is a very powerful laser pointer.) Over the course of the next minute, you rotate it through three quarters of a circle, hold for ten seconds, then turn it off.

For twenty more minutes, nothing happens to the dot. Then, it starts moving steadily across the wall, somewhat slower than the second hand of a clock, but seemingly moving over forty times the speed of light! A minute passes, and the dot sheds its impossible speed just as easily as it picked it up, then, ten seconds later, finally disappears.

The key reason this is not a problem for our current understanding of physics is that being next to the dot at one moment gives you no information about the pace where the dot just was previously; it's all completely different light.

Anyway, let's give it another shot. Maybe this time I'll have the patience to wait for him to quote or namedrop someone.

Aww yeah, "NJ Wildberger" (51). This is now a Norman Wildberger discussion blog. I will still quickly explain what Terrence is getting wrong here. Wildberger is a finitist; among other things, he doesn't feel that it makes sense to talk about irrational numbers, where part of his concern is the lack of an exact decimal representation. Terrence brings up the square root of two here, and I feel the only way to do justice to Wildberger here is to enumerate the three different viewpoints in play here:

Most mathematicians consider the square root of two to be an irrational number.
Wildberger accepts at least one proof that there is no rational number whose square is exactly two.
Terrence categorically rejects all proofs that there is no rational number whose square is exactly two.

In my capacity as a CS (Computer Sicko), I have a lot of interest in Wildberger's work in finitist mathematics, and I'd like to be better read in that area than I currently am. This is because it's helpful when a computational procedure terminates in a finite (ideally small) number of steps, and provides an accurate answer. This opinion needs to be qualified with the caveat that although I think of these as practical concerns, I haven't actually written code based on Wildberger's work, so I can't honestly say that my motivations here are, as yet, anything other than aesthetic.

With that said, the idea of developing a discrete geometry, as I understand Wildberger has done, is neat, in the same way that I was saying that semirings look neat. At the same time, I'm not interested in restricting myself to finite mathematics, and I'm now remembering my attempts to work with Dafny. Perhaps I'll look into that again later.

Anyway, I think I'm done for tonight.

Good night.

Terryology 2024-11-08

2024-11-08T05:00:00-05:00

While I could keep making fun of Walter Russell's element charts (and I kind of want to), I think the more interesting thing I'm seeing in his writings are references to action and reaction, references to Newton, which Terrence Howard repeats. What's interesting to me is how the reference to Newton kind of doesn't work. In the context of Newton's third law, action and reaction are balanced forces which occur simultaneously. According to Russell, action and reaction are successive motions. These are a different kind of physical quantity, and there isn't an obvious mathematical relation between action and reaction under Russell's (minutes-long pause as I decide what to type) philosophy.

So, let's see what kind of stuff Terrence Howard does with this. On page 19, we see him describe "1 x 1 =(ing) 1" as "ACTION times ACTION without a REACTION". Now, I didn't read too much of Russell, but it seems clear to me that the numerological meaning that Terrence is assigning to the number 1 is basically the same as Russell's idea of a single, indivisible mental substance that creates the appearance of our reality through thought. However, associating it also to "action" represents, I think, a break with Russell. From Russell, I see that One is mind, thought is motion, motion is action and reaction. From Terrence, One is action and reaction, and presumably therefore motion, so I'm not clear on how Terrence's ideas could relate to Russell's concepts of "stillness" or "inertia".

Anyway, I decided to read a bit further ahead, and I noticed something kind of amusing. On page 38 (both versions), Terrence refers to the cover of the book, and the question it asks. This is basically comprehensible in the earlier version, where the cover does have a question on it, but the later version doesn't have any text, punctuation, anything like that. The editor (I'm still baffled that there allegedly is one) asleep at the wheel as usual, I see.

Although that question, being about money, does get at the heart of one of the darkly humorous aspects of this whole endeavor. Paraphrased:

"I think the bank is ripping me off." "Really? What makes you suspect that?" "You see, the transcendent nature of mind and being..."

Also, why is it that Terrence is so concerned about, in essence, conserving the number of symbols on each side of an equals sign (4, 18, 19, 49, 52) but claims that matter can spontaneously generate (35)? I mean, when it comes to ice, it's "just" that he's drawing a distinction between water and moisture for some reason, but he does in fact seem to be saying that hydrogen will simply appear in a sufficiently star-like environment.

That's all for now; I'm feeling kind of egh right now.

Good night.

Terryology 2024-11-07

2024-11-07T05:00:00-05:00

One thing I want to look at about Terrence Howard's beliefs is the way that many of them didn't form in a vacuum. Aside from some annoyingly common beliefs, he also, let's say, "cites", previous works of pseudoscience by other people, as well as fringe or niche areas of mathematics that he frankly does no favors by mentioning.

Hold on, before I get into that, let's take a moment to think about "citation". Terrence Howard's book, apparently called One Times One Equals Two has a strange layout, and there are multiple versions available.

I was previously working from a version I downloaded from a now-defunct Flat Earth website (which is amusing, because he explicitly believes in a spherical Earth with atmosphere fading into vacuum), which uses various pictographs on the title page, and has the mirror image of the title page as the back cover. The version I'll be mostly using this month has an abstract wash of color for the title page, with a flower in the upper middle, and a symbol in the lower middle. Each version has an end page with a date, and the Wash Of Color version has a later date (September 23, 2020) than the Pictograph version (May 19, 2019).

The pages are numbered starting from 1 on the title page. Most pages have a pair of columns that look like pages, but the page number is outside and between the columns. There is little enough on a given page that it should usually suffice to simply cite by page number.

With all that said, let's see what jumps out at me.

Hmm. I've actually got some mathematical ideas from looking at this. Probably not novel, and I know of some prior art that's pretty close. On page 15, Terrence makes some comments about the act of multiplying 1 by 1, which are entirely fatuous in the context of standard mathematical abstractions. Talking about how the 1's are things that have to be conserved. I thought to myself, having apparently not gotten this out of my system yesterday/two months ago,

"Oh, come on. If you do nothing, and then you do nothing again, then overall you did nothing. ... If you do nothing for a minute, and then do nothing for a minute again, then you did nothing, but. But it took two minutes. Which we don't care about. But what if we did?"

Suppose we have a magma, and we pick out a set of generators, which need not be minimal or finite, and to each generator in the set we assign a cost. (For reasons of elegance, the identity element, if any, should have a cost of zero unless there's a specific reason to give it a higher cost.) When we apply the monoid operation to two elements with associated costs, we add the costs together, to produce the cost associated with the result. We can divide all possible (element, cost) pairs into equivalence classes based on the element, and take the one with the smallest cost in a given class as representative. Fans of recreational mathematics will recognize this as one way of thinking about "God's number", an idea that I imagine will sound interesting to Terrence until he realizes how much it has to do with cubes.

I can imagine various variants of this idea, such as using sets of generators instead of numeric costs, associating each generator to the set containing only itself, using set union instead of addition, and creating a function from each element to the subset of the power set of generators associated with it. The other thing that comes to mind is to look for a semiring that would do something interesting if you took the direct product with the tropical semiring. Actually, semirings in general look neat. I should mess around with them.

Anyway, let's see if anything else jumps out at me.

The next thing I ended up looking into has a lot more to dig into, and I'll see what I can find out about it later. Basically, I got to page 32, where he reproduces Walter Russell's goofy circle element chart, and I decided to look into it some. I took a look at The Universal One, and I have to say, these takes on Mendeleev's 1904 revision of the periodic table are fascinating, and not in a good way.

Before the discovery of the noble gasses, elements were grouped into "octaves" of seven each, with the idea that the elements regularly cycle through different properties as atomic weight increases. This concept informed the development of the periodic table; at one point, the groups of the periodic table corresponded to the different members of the octave, with an additional group for the "triad" that separated some pairs of octaves, and was seen to mediate between them. When the noble gasses were discovered, they were originally put on the left, with a gap in the group following every period that ended in a triad. The resulting jagged chart looks strange to us now, but the main concern at the time was, like, copper shouldn't be classified as an alkali metal, right? Like, that just doesn't make sense.

Anyway, Walter Russell, writing presumably sometime after 1920, criticized the 1904 revision, which, like, could he have done a later one instead? I get that he didn't have a search engine... Anyway, he declared that there were "missing" noble gasses (which would presumably go somewhere between group 8 and group 10 in the modern table), but also that various elements were in the wrong place. He disagreed with the existence of Group VIII (groups 8 to 10 today), but some of his other decisions were... confusing, from a modern perspective. He takes no issue with the equivalent of the first three periods (understandable, as they fit with the octave concept that he's trying to revive), and then he gets partway through the fourth period, gives his blessing to scandium (modern group 3) as being in group III (which until then had corresponded to modern group 13), and then highlights everything from titanium to germanium as out-of-place, including gallium, which would actually make sense to go under boron and aluminum. He does generally keep up this pattern, so I guess he's being consistent...

He also calls out large gaps in the lower rows. Uh, yeah, Walter, that's because those elements hadn't been discovered yet. What were they supposed to put in those spots? (Putting aside that the lower periods were doubled up, and the lanthanoids and actinoids hadn't been separated out properly.) Basically, this chart had problems, but it got important things right, and Walter Russell seems to have had an amazing instinct for getting those categories exactly backwards.

Reading the chapters around that caption for context, I can see a lot of the buzzwords that Terrence tends to pull out.

Anyway, I'll see where my interest takes me next. For now, if I don't get ready for bed, I will regret it.

Good night.

Terryology 2024-11-06

2024-11-06T05:00:00-05:00

I write this a few months before I plan to publish it. As the US presidential election bears inexorably down upon us, I find myself remembering how I distracted myself in the weeks following the last one: reading the nonsensical work of one Terrence Dashon Howard.

Now, ever since his appearance on The Joe Rogan Experience earlier this year, it's been fashionable, though by no means new, to try to explain the problems with Terryology from a mathematical perspective. People were jumping into his Twitter replies the moment he published a "paper" in a tweet, and my later attempts to explain group theory for SoME were inspired by his failure to understand why particular fundamental concepts of groups are the way they are.

However, in the years since, I've come to realize that such critiques cannot hope to convince Terrence himself, because they buy into a fundamental misunderstanding on Terrence's part. In short, he cannot currently be convinced of the falsity of 1 x 1 = 2, because he is using literally every symbol in that statement with a different meaning from accepted mathematical practice.

That said, I still feel the urge to try, so I'm going to try to get it out of my system:

Oh, I guess I'm doing this, to set it off from the rest of the post.

Ahem.

Mr. Howard, suppose you own a bank operating according to the principles of Terryology. One day, a customer comes in with two checks made out to "cash", from two of your account holders. The checks are entirely legitimate, and the accounts have enough money to pay out the respective checks. One check is for $2, and the other check is for $3. How much should the teller pay the customer, and why?

The purpose of this seemingly trivial question is straightforward: by mainstream mathematics, the teller should pay the customer $5, but by analogy with your insistence that "1x1=2" because $1 and $1 is $2, the Terryological answer would appear to be "2x3=6". If you do not accept that the teller should pay $6, then I would like to know what $1 and $1 making $2 has to do with multiplication, and if you do accept it, I would like to give you a $10 bill, a $10 bill, a $10 bill, a $10 bill, a $10 bill, a $10 bill, a $10 bill, a $10 bill, a $10 bill, and a $10 bill, in exchange for a check equal to one tenth of their total value.

Remember, Terrence, it was the smiley that tried to swindle you out of a billion dollars, not me.

Anyway, the fundamental incompatibility between Terryology, and, um, real math, can be expressed philosophically:

Because nobody knows all mathematical truth ahead of time, mathematicians have to accept that they have incomplete knowledge, and develop tools to compensate for that, either by deriving new knowledge from existing knowledge, or by finding new ways to use existing knowledge. For the former, we can look at stuff like Euclid's elements, and for the latter, we can consider the technique of checking a calculation by casting out nines. The latter case is an example of abstraction, and it's a powerful tool: find the right abstraction to represent a problem, and you can come up with a general answer without getting bogged down by the specifics.

By contrast, the objects in Terryology are in some sense un-abstractable. An idiomatic rendering of Terryology's 1 x 1 = 2 would be something like "when considering the totality of existence, we may compound it with itself, and this produces a [duality?]". The concepts being represented are in some sense atomic.

Basically, this isn't mathematics, it's numerology.

I took some time (not a lot...) to polish this post, but my planned followup posts are going to be a bit rougher.

Good night.

Weekly Roundup 2024-11-05

2024-11-05T05:00:00-05:00

Wednesday: I gave up on figuring out if orbitals are possible per se in four spatial dimensions, and decided to just focus on understanding what they would do if they were.
Thursday: I made some progress understanding this stuff, and came up with various ideas for further investigation.
Friday: A little progress, not much.
Saturday: A little more progress.
Sunday: Sleepless agony, an entire day spent shielding my eyes and desperately seeking rest.
Monday: I'm feeling fine now, and here are some ideas related to organic chemistry.

Since the last entry, I've decided to refer to this hypothetical setting as the Clifford Universe, in contrast with our own Axial Universe.

Next week, I'm going to take a planned swerve, and it's going to go on as long as it needs to.

4D Worldbuilding 2024-11-04

2024-11-04T05:00:00-05:00

Sleeping eventually cured everything that was wrong with me yesterday, so now I'm pondering one of the tricky bits of any "making up chemistry" endeavor:

Now that I've got an oxygen equivalent, I've got a hopeful water equivalent.
I'm confident but not certain of the bond structure in the water equivalent.
I have no idea whether any possible bond structure allows for a less-dense solid phase, or what parameters to tweak to get there.

On the plus side, I do have a basic picture of how organic chemistry "should" work. Where three-dimensional organic chemistry gets a lot of mileage out of hydrogen, carbon, nitrogen, and oxygen, the stuff I've worked out in four dimensions has five elements in the place of these four, with rough equivalents to each, plus one that acts sort of intermediate between nitrogen and oxygen. The additional element is necessary because of the greater number of bonds available in four-dimensional space.

So, the more oxygen-like one would create ethers with two functional groups, but ketones with three, while the intermediate would create ethers with three functional groups, but ketones with two. (Or maybe the ethers wouldn't be as important, considering that I didn't find "ethers but with nitrogen instead of oxygen" when I looked just now. But there would be two kinds of amide, probably. Or, various kinds of amide. Oh man, I'm imagining all sorts of horrifying bond structures now...)

Anyway, I should get ready for bed, and to shift gears in a few days...

Good night.

4D Worldbuilding 2024-11-03

2024-11-03T04:00:00-05:00

I've got an amazing amount of ailments and status effects simultaneously, so instead of trying to accomplish anything in particular today, I'm going to try to focus on curling up in a ball and feeling sorry for myself.

That said, I have managed to do most of the initial work in devising the equivalents of important elements for organic chemistry. The next task right now is to nail down the proper equivalent of oxygen. I should probably stop for now, though. I seem to be making a bunch of math mistakes.

I'm going to post this way early and call it done.

Good night.

4D Worldbuilding 2024-11-02

2024-11-02T04:00:00-04:00

I'm splitting my time between working on the writeup, and taking notes that I need to talk cogently about the writeup. At some point, these notes are going to result in finally nailing down a rule-of-thumb for the relative strengths of particular kinds of bonds that can't exist in three dimensions, but I'm not quite there yet.

I've got a lot of stuff going on that's making it a bit hard to think, so I'm going to bed.

Good night.

4D Worldbuilding 2024-11-01

2024-11-01T04:00:00-04:00

I've got enough of an understanding of molecular orbitals (mediated by my aesthetic preferences) that I've started working on a writeup, but it wasn't long before I got to the point where I'm going to need to actually draw the diagrams instead of holding them in my head.

As far as stuff to write about in this post... eh, it didn't really work out that there was much more than what I already wrote. Not going to push things.

Good night.

4D Worldbuilding 2024-10-31

2024-10-31T04:00:00-04:00

Here's where things stand:

I've resolved my confusion about sigma orbitals, and they now make much more sense to me.
I've figured out how to make sense of pi orbitals in the context of the weird things bonds can do in four dimensions.
This should be enough information to analyze the valence shells of various arrangements of pure elements.
The trends related to electronegativity aren't too confusing, though I haven't run through the previous steps in enough detail for it to be worth it.
I think I understand the basic magnetic implications of the different kinds of partially-filled orbitals.

Here's where I think I need to go next:

Write out the molecular orbitals for the first and second rows.
Characterize the pure elements of the first row.
Characterize the pure elements of the second row.
Sketch out electronegativity; figure out what molecular orbitals "intermediate between" the above orbitals look like.
Do a basic sketch of the properties of the first row of transition metals.
Get some basic idea of what kinds of ions and crystals can develop.
See if I can figure out what's going on inside the nucleons, because if I'm understanding the very basics of QCD correctly, naively scaling it up to four spatial dimensions would cause... problems. Problems which I have already presumed do not occur. (Like, anti-muons kind of seem like a safer bet than protons, but maybe there's something really obvious that saves protons or breaks anti-muons.)

Somewhere in there, I'll hopefully get some sense of the equivalents of organic and biochemistry.

Anyway, I'm tired and spacing out, so that's enough for tonight.

Good night.

4D Worldbuilding 2024-10-30

2024-10-30T04:00:00-04:00

So, I ended up not crunching the numbers on solenoids, but I did crunch the numbers on other things.

First off, I did figure out how to scale up the Stern-Gerlach experiment, but I'm not totally sure what kind of square I expect it to create, just that it is a square.

I then felt comfortable synthesizing a bunch of Stack Exchange answers and Wikipedia articles into a probably-good-enough guess about electron orbitals and nuclear shells. The electron orbitals were independently checked by considering the number of spherical harmonics in 4 dimensions.

At this point, I'm feeling okay just ignoring the question of how the atoms are fitting together (people have tried to answer this elsewhere, but I haven't had the wherewithal to make sense of the answers), and forging ahead with sketching out chemistry. There are some things that I grasp better than others.

I've been pondering molecular orbitals; initial ideas about sigma orbitals are... confusing.
Until I grasp all of the ways that sigma orbitals interact, I don't think I have a hope of analyzing the second row.
Hopefully pi orbitals aren't too much harder.
I haven't reviewed electronegativity at all, so I'm not really ready to consider stuff like "what happens when a halogen-analogue bonds to anything else?"

Anyway, I let things go late and I don't think I have anything else to say now, so here's to drawing some painfully elaborate diagrams tomorrow.

Good night.

Weekly Roundup 2024-10-29

2024-10-29T04:00:00-04:00

Wednesday: I tried to remember taking partial differential equations.
Thursday: I tried to compensate for not really remembering taking partial differential equations.
Friday: I got something done, I forget what.
Saturday: I read about stuff like the Hamiltonian, and completely failed to remember the relevant classes.
Sunday: I decided to press on at a slightly higher level, finally understood a key aspect of the Stern-Gerlach experiment, and started crunching the numbers describing electromagnetism.
Monday: I tried to explain my thought process for that number crunching.

Next week, a bit more of this, before, well, you'll see...

4D Worldbuilding 2024-10-28

2024-10-28T04:00:00-04:00

Hm.

I haven't done too much more work on understanding higher-dimensional magnets. Let's just take what I was saying and try to lay it out without running the numbers yet.

I'm supposing that if I form a solenoid by coiling wire around four facets of a hypercube, then there are two kinds of facet that result: "pole" and "coil". My current intuition/belief is that I can get some idea of how various orientations of solenoids interact by enumerating the different relative orientations, given a fixed displacement, and then running the relative numbers.

I'm not sure how to look for an explanation of this stuff, so my hope is that I can find a geometric algebra expression by doing the calculations in three dimensions, seeing what kind of mathematical behavior I get in known physical arrangements, and then scaling them up. Before considering two solenoids next to each other, I should get some idea of the gradients around a single solenoid. It seems to me that a pole cell should change bivector orientation as you rotate around the face that it shares with a coil cell, but probably no change around the faces it shares with the other pole cells, but maybe my "intuitions" here are getting pushed too far?

It's possible that I'll need to take a break from this and come back to it when I'm ready to be a bit more focused and methodical, but either way, right now, I need to get to sleep.

Good night.

4D Worldbuilding 2024-10-27

2024-10-27T04:00:00-04:00

Here's where I'm standing, as I decide to finally give this stuff its own category.

As long as I'm assuming that the 4D world I'm coming up with has atoms that basically resemble the atoms of our own universe, then it stands to reason that I can come up with thought experiments that correspond to well-known historical experiments, such as the Stern-Gerlach experiment.

The Stern-Gerlach experiment concerns the use of an inhomogenous magnetic field to split a beam of atoms by acting on the magnetic moments of unpaired electrons. Under the influence of the field, the atoms are "measured" as pointing in the same overall direction as the field (in which case they are attracted to the greater field strength), or in the opposite direction (in which case they are attracted to the weaker field strength).

The Stern-Gerlach experiment features three mostly-orthogonal directions:

The beam travels "forwards"
The magnetic field lines point generally "up/down", but with an angular deflection so that the lines are more tightly spaced at one pole than at the other
This deflection appears in the "side-to-side" view of the field lines

It is key to note that the deflection in the experiment is, near as I can tell, not trivially related to stuff like the right-hand rule. The atoms are electrically neutral, so this is most simply thought of as different magnetic objects and fields interacting with each other.

Now, the Stern-Gerlach experiment demonstrates quantization of angular momentum in electrons, and one would presume that a similar experiment in higher dimensions could show something similar. But designing such an experiment has so far been... not easy.

There are two basic challenges I've been dealing with so far:

There are six components of the magnetic field in four dimensions; it's not a straightforward 4-vector
I'm not sure that magnetic "poles" exist, as such, in four dimensions

Let's start by looking at stuff about the latter; it may help to explain the former.

Instead of considering a permanent magnet, let's imagine various solenoids in increasing numbers of dimensions.

In two dimensions, we can create a solenoid by wrapping a square in a loop of wire, joining the ends, and declaring by fiat that a current flows. This creates a scalar magnetic field within the square.

In three dimensions, we can create a solenoid by wrapping four faces of a cube in coils of wire, leaving two opposite faces exposed. There are three possible choices of uncovered faces, corresponding to the three components of the magnetic field. At this point, it would be helpful to leave behind the concept of the magnetic field as being made of vectors/lines, and thinking of it in geometric algebra terms, as a bivector that deflects moving charges to the degree that their motion is coincident with it. The bivector is perpendicular to the "traditional" vector field, forming a manifold surface that emanates from moving charges, and the like.

So, when we take all of that and move it up to four dimensions, we can consider wrapping a very long coil around four of the cubic cells of a tesseract. There are six possible choices of four cells, corresponding to the six bivector components of the four-dimensional magnetic field. For a given choice of cells, the solenoid's current will be flowing generally within a single plane, producing a bivector aligned with that plane, with one orientation in the interior of the tesseract, and the opposite orientation outside of it. The magnetic field must smoothly change in some way for a trajectory through space, to get from one orientation to the other.

In three dimensions, this can be seen with the curving field lines that connect the poles, or, equivalently, with the changing orientation of the bivectors for an arbitary path from inside the solenoid to outside. Key to our three-dimensional intuitions is the fact that the two pole faces are isolated on the cube, unable to reach each other over the surface without passing through the coil.

This latter property does not hold on the tesseract; the open faces form a single region. There is a path from any cell, to the opposite cell, and the bivector must vary smoothly between then. I don't have a grasp on this yet, but I'm going to try.

Key facts include:

Each coil cell shares one coordinate with the magnetic bivector produced by the coil, and its other two coordinates are perpendicular to the coil.
Conversely, the "pole" cells have both vectors that went into the bivector, and one other perpendicular one.

Consider the sum of the bivectors at the center of a "pole" cell. Idealize the coil down to a single loop. The sum will have four components, one for each coil cell.

Suppose the tesseract has side length of two and is centered at the origin, with the coil in the wx plane. Therefore, we can, without loss of generality, put the point of interest at (0,0,0,1), and see how it interacts with the following line segments at their midpoints:

(-1, -1, 0, 0) to (1, -1, 0, 0) at (0, -1, 0, 0)
(1, -1, 0, 0) to (1, 1, 0, 0) at (1, 0, 0, 0)
(1, 1, 0, 0) to (-1, 1, 0, 0) at (0, 1, 0, 0)
(-1, 1, 0, 0) to (-1, -1, 0, 0) at (-1, 0, 0, 0)

This results in the following bivector products:

(1, 0, 0, 0) * (0, 1, 0, 1) = wx + wz
(0, 1, 0, 0) * (-1, 0, 0, 1) = wx + xz
(-1, 0, 0, 0) * (0, -1, 0, 1) = wx - wz
(0, -1, 0, 0) * (1, 0, 0, 1) = wx - xz

Summing together, these eliminate everything but the wx component, and it's relatively obvious that things would cancel out the same with the y coordinate instead of the z coordinate.

I'm very unsure of what to make of this, so let's try taking things down a dimension...

(1, 0, 0) * (0, 1, 1) = xy + xz
(0, 1, 0) * (-1, 0, 1) = xy + yz
(-1, 0, 0) * (0, -1, 1) = xy - xz
(0, -1, 0) * (1, 0, 1) = xy - yz

So, we get the same result of the other components canceling.

I guess what I need to take away from this is that to get an idea of the behavior of the field outside of the solenoid, I need to actually consider points outside of the solenoid.

To do that, I'm going to need to deal with the effect of distance on the field. I think in four dimensions it drops off quadratically, but I'm not sure, and right this moment, I'd be better served to sleep than to research this further.

Good night.

Diary 2024-10-26

2024-10-26T04:00:00-04:00

I've got a handle (mostly) on the scalar wave equation for four dimensions, so I decided to see if I could get any better of a grasp on the intricacies of atomic structure.

...

The answer is no.

I'm at a bit of a crossroads right now. I can either:

Arbitrarily pick orbital behavior that I like, and document the results
Try to figure out what a purely repulsive structure would look like (shards of a particular lattice structure, I guess...)
Put this down and mess with another project

I'm going to sleep on this for now.

Good night.

Diary 2024-10-25

2024-10-25T04:00:00-04:00

I made a little progress on the Huygens' principle stuff, but not enough to merit the tag. I just kind of took things easy today. And now it's late, and I should get to bed.

Good night.

Diary 2024-10-24

2024-10-24T04:00:00-04:00

Still reading up on Huygens' principle and it not applying. I'm a little lazy to do all of the necessary calculus right now, but I think I understand the notes I'm looking at well enough to kind of fake up having a solution for the evolution of an instantaneous signal in four dimensions.

One thing I've been trying to figure out, conceptually, based on these notes, is whether the behavior of the surface of a liquid is fundamentally different in four dimensions from three. Like, the implication from what I'm looking at here is that it should be in some sense "calmer" because the disturbances from an object entering the liquid wouldn't "linger". I could be totally misunderstanding so many things here, though.

Anyway, I spaced out and I should wrap up.

Good night.

Diary 2024-10-23

2024-10-23T04:00:00-04:00

I got distracted working on other stuff, but I did do some stuff with the 4D worldbuilding just now. I think I'm good with basic energy-momentum stuff, and now I need to take a look at waves, which is going to be interesting, because I haven't yet found an easily digestible explanation of what it means for Huygens' principle not to apply in an even number of spatial dimensions. I'm sure this will have numerous consequences, I'm just not sure what they are yet. Hopefully I'll be able to explain them once I understand for myself. Anyway, that's not happening right now.

Good night.

Weekly Roundup 2024-10-22

2024-10-22T04:00:00-04:00

Wednesday: Reading.
Thursday: Finished reading.
Friday: Thinking about higher-dimensional physics.
Saturday: A little progress with higher-dimensional physics.
Sunday: Nothing to report.
Monday: A little more progress on the physics, and some very oblique mockery.

Next week, I'll try to increase the pace on this worldbuilding stuff.

Diary 2024-10-21

2024-10-21T04:00:00-04:00

I'm making slow but steady progress writing up four-dimensional physics. Right now I'm still in the realm of "just a straightforward addition of a spatial dimension". This doesn't require much creativity yet, but it does require noticing that I'm somewhat abusing notation and so sometimes I can miss a minus sign if I'm careless.

As far as "creativity yet", I think it'll be a while before I have to consider the effects of attractive forces, but once I do, I will need to make some subjective decisions.

For now, I really want to get to sleep.

Good night.

Diary 2024-10-20

2024-10-20T04:00:00-04:00

Nothing to put here right now, because I was just taking things easy today. Maybe I'll have something tomorrow, maybe I won't.

Good night.

Diary 2024-10-19

2024-10-19T04:00:00-04:00

I had a kind of rough day today, so so far I've just made a tiny amount of progress on writing up the most straightforward way to have four dimensions; trying to follow what Dewdney called the "principle of similarity" when working on the Planiverse. Even if, as I suspect/worry, a lot of stuff will eventually need to be thrown out, the basic stuff I'm reviewing now should be pretty solid. For now, though, I need to get to bed soon.

Good night.

Diary 2024-10-18

2024-10-18T04:00:00-04:00

I've been thinking about some things that could be a proper project, but I think I'm biting off more than I can chew. (Actual quote: "What would this imply? I have no idea.") So, I need to either find something more manageable, or really properly break up the stuff I've mostly been thinking about.

Let's have a stab at the latter. I've been speculating about four-dimensional planets. Geography, anatomy of the inhabitants, rotation, etc. However, a big problem with this kind of world-building project is that in a universe with four spatial dimensions, bounded orbits aren't possible (unless there's some kind of significant positive spatial curvature, maybe), and for that matter, atomic orbitals aren't stable (the theory here is more complicated than with the orbits, and I'd like to run it through myself, just to be sure; that does to a strong investment in learning/relearning the relevant material). I want to have, like, a hyper-spherical planet with gravity and celestial bodies, but I can't get them naively. I can see a few ways to deal with this contradiction:

Ignore it.
Run through the relevant theories and see what kind of "little tweaks" are possible to try to salvage familiar behavior.
Toss nearly everything out, come up with another physical system, and try to derive stuff like life and rivers in the context of even more alien physics.

While I've made a few stabs at the third, I think I need something with a bit more of a guide rail. I'd like to try for the second, because coming up with a plausible set of physical rules would let me have some idea of how chemistry works. (And judging by what I've come up with under the assumption that four-dimensional orbitals can contain four electrons, and considering just the 1s orbital, I could draw so many crazy diagrams if I just had some assurance that every atom involved would not spontaneously destroy itself.)

Since the books I was reading were the Orthogonal trilogy, I'm right now thinking that I might as well reread the supplemental materials, and try to combine the stuff about the "Lorentzian universe" with the adaptations to four spacelike dimensions.

My initial goal there would be to see if I can confirm the statements that atoms aren't stable.

Anyway, that's not happening now; I need to get to bed.

Good night.

Diary 2024-10-17

2024-10-17T04:00:00-04:00

I'm through the current set of books I was reading, so I'm trying to kind of reorient and see if I have any particular writing I want to get back to. Whatever I decide on, I'm probably not going to settle on it tonight.

I'm just going to wind down right now.

Good night.

Diary 2024-10-16

2024-10-16T04:00:00-04:00

I was still kind of a blob today. I've been reading books. When I finish the current one, I'll try to make a decision on whether to pick out another, or to get back into project stuff. For now, though, I really need to get to bed.

Good night.

Weekly Roundup 2024-10-15

2024-10-15T04:00:00-04:00

Wednesday: I got a little bit done.
Thursday: I got a little bit more done.
Friday: I tried to write stuff for a PR to Jinja, but I got confused by the test structure.
Saturday: I didn't really get anything done.
Sunday: Traveled.
Monday: Traveled.

Next week, I'll see if there's stuff I'm up for doing.

Diary 2024-10-14

2024-10-14T04:00:00-04:00

And we're back. The return trip was not as bad.

I just kind of vibed today, nothing worth mentioning. So I'm going to wrap up and get to bed.

Good night.

Diary 2024-10-13

2024-10-13T04:00:00-04:00

We were on the road today, and it was super rough. Nothing much on my mind, dealing with the drive.

Good night.

Diary 2024-10-12

2024-10-12T04:00:00-04:00

I had a rough day/week, so I'm taking things easy, and I don't have much to discuss currently.

I just want to zone out.

Good night.

Diary 2024-10-11

2024-10-11T04:00:00-04:00

Right now, I'm more feeling like reading than writing, so let's go with that for a bit.

Aside from that, though, I'm looking into something a little more tractable than MOTR: a minor issue in Jinja that really bothered me. I don't want to rush anything, so I'll try to ease into writing tests, but, ugh, these tests require more easing-into than I'd like. Maybe they'll make more sense if I study them in more detail, but I'm not feeling confident yet.

Anyway, I'm tired and I want to wrap things up early.

Good night.

Diary 2024-10-10

2024-10-10T04:00:00-04:00

I've come up with a working title for the solo RP stuff (Twisted Knight), but I haven't done much else with this or anything else...

My urge right now is to take some time to rest, then try to remember what the heck I was doing with MOTR. If I'm lucky, I can get a release out this year.

Good night.

Diary 2024-10-09

2024-10-09T04:00:00-04:00

I've got a good idea where I'm going with the solo RPG stuff, but what's really missing right now for me is a name so I can give it a category here.

On the other hand, I want to get beyond my "session zero" before I start talking about it too much, so I guess it kind of evens out for now.

I'll take things easy.

Good night.

Weekly Roundup 2024-10-08

2024-10-08T04:00:00-04:00

Wednesday: I worked around a bunch of bugs that sound extremely fake, but I've got reproductions, so oh well.
Thursday: I worked around some issues that were somewhat more of my own making.
Friday: I decided to put Stars Without Number aside to focus on other stuff for a change.
Saturday: The "other stuff" was worldbuilding in 4D, and I had a plan for getting bounded orbits to "work".
Sunday: The plan didn't really work out; unless I'm missing something, I can't use it to make the orbits "more than slightly bounded" without messing other stuff up.
Monday: I put that aside (although I did come up with something that I think has potential) in favor of a new project, that I'm hoping I manage to make some progress on.

Next week, I'll try to keep up that writing, although the other RPGs I was looking into kind of have me wanting to bring some other code I wrote into the project, but that code is just weird enough that I'd have to write actual tests for the existing code before I considered it... Best to leave that idea be for now.

Diary 2024-10-07

2024-10-07T04:00:00-04:00

I spent a lot of today looking for a solo RPG that looked like something I'd like right now. I came up with a few ideas that could have potential, but probably work out to procrastination in the short term.

After weighing what I found, I decided to take a slightly different tack to the solo campaign I've been writing code for. I'll shelve the Stars Without Number campaign for a bit, and focus on running something with just Mythic. (Specifically, just the second edition core for now. Reading the magazine is procrastination.)

I'm right now trying to properly interpret all of the stuff I rolled for the main character. I could have kept on with that tonight, but it's getting late, so I'll work on it tomorrow afternoon I guess. For now, sleep.

Good night.

Diary 2024-10-06

2024-10-06T04:00:00-04:00

The numbers have been crunched, and I'm pretty sure they look... not great. I'm going to have to shelve this 4D stuff, at least from a cosmological perspective, for now.

I mean, from a writing perspective, maybe I can make "where is the light and energy coming from?" into a mysterious question, but I'm not feeling that right now.

Ugh.

I want to either make some more progress with this, or find something else to focus on. I'm going to wrap up now and do some reading.

Good night.

Diary 2024-10-05

2024-10-05T04:00:00-04:00

I'm still not ready to give this a dedicated category, but this is what I was up for working on after my allergies laid me out.

Hold on, didn't you also—

Shh, not now.

You can't hide from your sins forever.

The world isn't ready to see how I got subclassing to work with Enum and frozen attrs classes.

Anyway, I'm still thinking about 4D stuff. I came up with a way to cheat the gravitational potential issues, but I haven't run the numbers yet to see if I like the behavior.

Basically, I need to take some derivatives, find the circular orbital distance as a function of the angular momentum, then take the derivative of that, and find the equivalent numbers for 3D space to compare how sensitive the different formulas are to perturbations in angular momentum. Like, intuitively, it seems possible that my "cheat" could result in orbits that change significantly in response to tiny nudges.

Anyway, it's late, I'll crunch those numbers in the morning.

Good night.

Diary 2024-10-04

2024-10-04T04:00:00-04:00

Okay, I do not want to keep on with the Stars Without Number stuff right now. I'm just too overwhelmed.

I've got a worldbuilding project that I'm working on, but right now I'm trying to deal with a technical hitch that's threatening to undermine the whole idea. (Basically, I'm trying to work with a 4D planet, and 4D objects... can't orbit. Or... two 4D objects can't orbit? I just found some stuff on ArXiv that looks too good to be true, but that's better than my previous sources of "too good to be true".)

I'll see what I feel like tomorrow. Right now, I'm just letting some of this stuff, like, wash over me.

Good night.

Coding 2024-10-03

2024-10-03T04:00:00-04:00

Okay. Now that I've been through a whirlwind of print-debugging Jinja templates (bleh), it's time for me to make sure that my CSS stuff works by actually writing styling code.

I hope you can appreciate how this is inducing a slight feeling of whiplash. Anyway, let's start off with setting a font and some color stuff. Good thing I specifically documented some bits of the theme-related code, or I'd be really annoyed trying to remember what's what.

...

Huh. That didn't work.

I guess

I guess I'll just have to insert some print calls—

Sorry folks, minor technical difficulties.

...

Okay, I'm better now. Probably. We just need to run the command again. What's the worst that could happen?

We'll never know because it worked! Second first try! Yeah!

Okay, now that we know that this actually works, it's time to do some actual styling.

...

OW!

What?

I keep

stepping

on these

rakes!

...

Okay, I got it. I had this weird multi-layered collection of attrs classes wrapped up in an enum, and because I almost always freeze my attrs classes, I couldn't subclass both the attrs class and the enum containing the instances, which meant that the enum values didn't have access to the attributes on their values. I tried to solve this with __getattr__. I do not recommend solving such problems with __getattr__.

In the end, I just had the one bit of code using this class add a .value to get the underlying attrs class instance.

Then that all worked fine, and I could see that the generated PDF had an unsightly margin, so I put in some code to get rid of that sort of by fiat.

After all of that, here's where things stand:

The header should be centered.
The header should probably be bigger.
I haven't tried to do anything to fix the map's styling.

I think after those previous... reactions, I need to take some time to cool off. Well, I'll center the header. That's easy.

...

What?

It actually was easy.

But I'm not going to push my luck.

Good night.

Coding 2024-10-02

2024-10-02T04:00:00-04:00

I have some things to complain about in the journey that I took to get here, but at this point, I now have to do the following things, down from the list two days ago:

Write styling code.
Figure out how to put text where I want it in SVG.

I would have gotten everything else done more quickly if it hadn't been for two weird bugs I hit. Basically:

Jinja doesn't seem to want to remember variables named self when it's working with contexts.
Creating an enum.Flag member with value 0 in a base class appears to work, until it very obviously doesn't.

I've filed a bug on the former, and have worked around it with a mixture of hand-written {% self_ = self %} and injected {% self = self_ %} blocks.

I haven't filed a bug on the latter, because I'm not totally sure how many bugs I'm even looking at, but I've devised a workaround based on forcing _member_map_ to be truthy.

...

And I've just discovered that that workaround doesn't quite go far enough.

...

There. Now it's better.

Anyway, I should have finished this entry before I started it.

Good night.

Weekly Roundup 2024-10-01

2024-10-01T04:00:00-04:00

Wednesday: I was still thinking about how to put together Jinja, svgwrite, and my hand-rolled weird code.
Thursday: I thought about CSS, and which classes I think I should have.
Friday: I started thinking about how the different "Components" that the BEM code I've written should look.
Saturday: I refined those ideas, but not all the way.
Sunday: I finished laying more groundwork, thereby preparing me to... lay more groundwork.
Monday: I figured out what I still need to do from a technical, move the data around perspective, but that means that once I'm done with that, I need to properly understand SVG styling, and the weird stuff I've done in other posts here notwithstanding, my grasp on the relevant fundamentals is not where I want it to be.

Next week, I'll try to get all of those tasks from yesterday handled.

Coding 2024-09-30

2024-09-30T04:00:00-04:00

I got to work on the stuff I was talking about last night. At this point, the stuff that's missing is:

Write a component-based HTML template.
Move the CSS template to the base location.
Ponder whether I need to be using an overlay environment at all at this point.
Write styling code.
Figure out how to put text where I want it in SVG.

That last one is somewhat more complicated than "Just specify the coordinates, what's the problem", because I want to have some offset between the bottom of the text and the bottom of the hexagonal cell, and I want the text to be centered horizontally relative to the hex.

I haven't really looked into the details of SVG layout and transforms, and I'm hoping that there's a declarative way to handle this, or at the very least something that means I don't need to know the height and width of arbitrary text.

...

Hm, looks like I want to do something with text-anchor and maybe dominant-baseline. ... Huh... It looks like I made some bad assumptions about how text works in SVG, which should mean that this will be easier than I feared, but I should still learn about dominant-baseline in the context of putting text elsewhere in the cell.

Anyway, I'm going to put all of that off until after I've done the rest of the stuff I listed above. And I'm going to do the rest of the stuff I listed above... later. Wind down properly now.

Good night.

Coding 2024-09-29

2024-09-29T04:00:00-04:00

I've been planning out and executing on refactors, so in terms of "features" or "getting the map to not look completely terrible", I haven't made any visible progress. At this point, the elaborate scaffolding I've written to put together BEM stuff automatically is as done as it'll ever be, so it's time for me to write component classes and make sure everything works the way I expect it to.

Basically, to start with, I want a DocumentComponent class that is a body tag, and has a tuple of bem.Component[Page] instances. Then, I can define a HexesPageComponent class, and get some good template stuff going on in there. Once those are written, I can try substituting them into the current template I'm using, and see what breaks. If I get that template working, I'll probably move it to a more accessible location. This all basically makes sense to me, but I'm too tired to explain it in detail, so I'm going to cut things off here.

Good night.

Coding 2024-09-28

2024-09-28T04:00:00-04:00

As I hinted in the summary yesterday, I ended up thinking about how to write component-y code with Protocols instead of ABCs. Let's see what I've got...

The default assumption for how to render the content of the main tag is "something involving Jinja", so the protocol needs to feature a method that takes an Environment and returns str | None. It also needs properties that produce str (tag name), Mapping[str, str] (attribute values), and a Modifiable instance (the super class of both Block and Element; I am making compromises with the design because "roll a compatible version of Enum that avoids using inheritance is Not Worth It).

My thinking is that I should make all of the processing for this into something that is available as a filter. That way the templates for a block can just use the filter to render the elements. Perhaps it would make sense for the value from the method to be Template | None? I'm going back and forth on this right now. The ultimate entry point for all of this is a filter call in the top-level template, and I'd like to be able to have that template enforce the tag used for the return from that filter. So, the filter looks something like def component(environment, value, require_tag=None): Or maybe this is written as a macro in a template. Possibly the requiring tag stuff is a filter and everything else is a macro. That feels like something that works acceptably...

I'm not up for thinking too much more about this right now, so I'll let it go for the moment and take things easy.

Good night.

Coding 2024-09-27

2024-09-27T04:00:00-04:00

Okay, here's what's running through my head currently. Blocks and elements are to be represented with Python objects. There is some way to render these objects to HTML. I am going to assume that there is one way to render a given block/element. Therefore, because the question of what is available while rendering is so tied up in the relevant class definition, I think I'll store templates in ClassVar[str] attributes, rather than in files. Common code should go in the template files, but I don't see that it makes sense to separate Python class definitions from template code, when they're so coupled. Figuring out how to make the proper template environments available is going to be a bit of a thing, probably, but it's fine.

What goes into rendering a template for a block/element? Well, the block/element object has to say what kind of tag to use, and provide any required attributes. It also has to provide class information in the form of an appropriately-typed flag object. Finally, the tag contents. Optional template string. If None, then there is no end-tag. (HTML has void tags but not self-closing tags.) If anything else, including the empty string, then it is rendered with {"self": self} as the context and inserted between the start tag and end tag.

I think this all makes sense; I'll have to try it out later and see.

Good night.

Coding 2024-09-26

2024-09-26T04:00:00-04:00

Okay, here we go again. I've got the code just about put together to generate stylesheets. What I'm missing is, in essence, a good idea of the kinds of stylesheets I should be generating. I'll try walking through my thoughts around this and see if I get anywhere...

I'm writing and styling HTML here, but the whole purpose of all of this is not to lay out anything for display on the web, but to put together a PDF document. So, I want to be thinking about this all as "a document" which is divided into "pages". "A page" will have content, stuff outside the content, and I guess that's it. The "outside" stuff will live on the document and get styled into the page.

Looking at it like this, I think this will be pretty flat from a BEM perspective. "Document" is a block, and many of the sub-elements will simply be elements directly under "Document". (Or maybe a "Page" is both a block and an element, or just a block, and all of the other elements live under Page? ... That probably makes more sense.)

If I assume that this is a good idea, then there needs to be some way to create these elements in a nested structure, and then pass that into the templating system to render it all to HTML. I've got stuff to handle the class names, but not the tag names. svgwrite generates XML...

I think I need to have the body class be a template argument, and the rest of the content is passed in as a kind of forest that the template just renders recursively-ish. I definitely need to take some more time to think about this. In what I have in mind right now, the different concerns seem either, not separated, or at least very jumbled up.

I'm going to leave it there for now.

Good night.

Coding 2024-09-25

2024-09-25T04:00:00-04:00

Hm. I haven't quite worked out the right way to integrate my BEM helper code with both Jinja and svgwrite. I've got something that probably works for Jinja, and something that might work for svgwrite. To properly test this, I'm going to need to write something to output CSS, which... might as well be in Jinja.

...

Anyway, I've spent enough time thinking about this and tweaking it for tonight. Tomorrow, I get to work un-stubbing the CSS function and attempting to actually style my HTML.

Good night.

Weekly Roundup 2024-09-24

2024-09-24T04:00:00-04:00

Wednesday: I got Click set up to automate PDF creation.
Thursday: I planned out some modelling for BEM concepts, and revived some old weird Python code in the process.
Friday: I worked on narrowing down the design some.
Saturday: I got into more detail there.
Sunday: I switched up part of the tech stack and improved on the BEM modeling design I've been working on.
Monday: I polished up some of the changes to the tech stack.

Next week, I'm hopefully going to have the energy and focus to evaluate how well the current set of dependencies works together.

Coding 2024-09-23

2024-09-23T04:00:00-04:00

I mostly took things easy today, so I once again don't have much to talk about, but I do have a few things I want to work on soon.

Here's what I need to do now:

Expand out the stub implementation of the CSS generation function.
Figure out how best to set up fonts.
Ponder how to inject BEM stuff into the template code (the SVG generation shouldn't be a big deal)
Test that I actually can style everything and have it look nice.

I think I should get things wrapped up and get to bed at a more reasonable time than I have been. That will help me get on this stuff in the next few days.

Good night.

Coding 2024-09-22

2024-09-22T04:00:00-04:00

I mostly thought about other things than Stars Without Number today (remember, that's what this BEM stuff is for), but I did make a few improvements:

Switched PDF output to Weasyprint, which I think is maintained.
Removed some of the limitations I initially designed into the Selector class.

With those done, I should only need some light tweaking to be ready to test this stuff out. For now, though, sleep.

Good night.

Coding 2024-09-21

2024-09-21T04:00:00-04:00

Breaking things down now, very carefully:

When I'm specifying the classes in a tag, I want to have access to a "typed subset" kind of construction. This is what the current Flag usage gets me.
When I'm specifying the classes in a CSS selector, I care very strongly about the ordering.

There are, fundamentally, three different cases to consider:

Unmodified block/element
Modified block/element
Element descendant of modified block

For a given block, the unmodified selectors should go before the modified ones, and the descendant tags can go anywhere because they have the highest specificity.

As such, let's try harmonizing ordering with how SCSS or suchlike would do it.

Unmodified block
Modified block
Descendants of modified block
Unmodified element
Modified element

My inclination right now is to do this with a partial ordering where only selectors with the same base block are comparable. To deal with the incompatible orderings, we need to sort them into an unordered collection of ordered bins, or rather, maps where we sort the keys before iterating. Regardless, they must be sorted by base type.

You're bringing back dependent mappings again, aren't you?

To be fair, there's only any actual code there if I bother trying to make it pass type checking.

Anyway, to get ordering working, I could try to coordinate stuff across multiple classes, or I could punt on that and just come up with some slightly strange ordering logic:

@attrs.frozen
class Selector[B]:
    block: B  # Require length of zero or one
    element: Element[B] | None  # Require length of zero or one

    # Define an ordering thus:
    # 0, None (bucket 0)
    # 1, None
    # 1, A0 (bucket 1)
    # 1, A1
    # 1, B0
    # 1, B1
    # 0, A0 (bucket 2)
    # 0, A1
    # 0, B0
    # 0, B1

It's a little funky that the block part has this alternation in the sort order, but I think it makes things work out aesthetically. Sorting happens first by bucket, and within buckets, sorts by tag name.

...

Okay, I did all of that, and maybe a few other things besides. I need to get away from this for a bit, rest and let the dust settle, before I work out if anything I just did actually made sense.

Good night.

Coding 2024-09-20

2024-09-20T04:00:00-04:00

Okay, it's late, so let's blitz through this.

After a bunch of really wonky metaprogramming, I have the beginning of a mini-library for manipulating BEM class names in Python. There are a few things missing, so I'll run through them quickly:

An arbitrary instance of the enums I've written should be able to generate CSS classes to go into a tag.
A one-element instance of the enums and either a zero-element instance or the class itself, should be used to generate CSS selectors to go into a style definition.

I'll try to figure this out later. For now, it's really late.

Good night.

Coding 2024-09-19

2024-09-19T04:00:00-04:00

I didn't turn on my computer until way late tonight, so I'm going to quickly talk about code I could write soon, rather than try to actually write it.

I'm still leaning towards using BEM for CSS. Now, the SCSS source for Three Dollar Quill doesn't quite adhere to BEM practices for a few reasons (one is that I have to write the code to work with HTML that I can't figure out how to easily control the generation of, another is that I was lazy and stuck everything in a single monolithic file). I will be again doing something weird, but for a different reason (I want to avoid setting up an asset pipeline, so I'm going to try to generate CSS in Python instead of Sass (although, for what it's worth, I could invoke Sass bindings from within Python...))

Anyway, all of that means that I think it would be neat to give Blocks and Elements a kind of programmatic identity. Like, here are the blocks, here are the associated elements, here are the modifiers they can take. And render them into various combinations. In class attributes, "here is a block or element, and a set of associated modifiers". In CSS...

A block
A block's modifier
An element
An element's modifier
An element that is a descendant of its associated block, with a particular modifier on that block

My heart keeps on wanting to represent the class attribute case with a flag enum, which I think indicates that there's something a bit wonky with my heart.

Like, there'd be two Flag subclasses that handle some behavior, and the Element one would need a reference to a subclass of Block, which means it's time for me to bust out the generic enum metaclass again.

🎉

Anyway, all of the CSS cases would require contracts on the length of the instance.

At this point, I've thought about this enough that I really want to try it to see what breaks.

Anyway, it's super late, time to get to bed.

Good night.

Coding 2024-09-18

2024-09-18T04:00:00-04:00

Okay, I've coded in some color schemes, and now I want to set up an entry point so testing this stuff out will be less obnoxious. I've got Click added in properly, but I need to figure out where to put it.

I'm leaning towards "a module next to the code it imports", but what to call it... Like, what kind of command do I want to be calling... [SOMETHING] campaign/sectors/sector.toml -o sector.pdf... export-sector, maybe?

...

And with a bit of futzing around, (as well as pointedly ignoring the deprecation warnings around some of the packages I'm using), I've got a nice command-line system for generating these files. With all of that out of the way, I can get to work improving the actual rendering logic powering all of this.

Anyway, I let things get late and I need to pack up and get to bed.

Good night.

Weekly Roundup 2024-09-17

2024-09-17T04:00:00-04:00

Wednesday: I decided to avoid using regular hexagons for my hex grid. They're close to regular, and can be made further if I desire.
Thursday: I generated an SVG hex grid; it had problems.
Friday: I started researching means of producing a PDF file instead of working with raw SVG.
Saturday: The solution I found uses HTML, which is fine, so I got my project set up to work with Jinja templates.
Sunday: Took it easy.
Monday: I generated a PDF from code; it had problems.

Bonus thing: I discovered today that some of the CSS on the blog was broken, so today's update is extra-big to fix it. (See, it pushes new versions of every page, to invalidate the CSS caching, and, eh.)

Next week, I'm going to try to put more of the pipeline together:

Take the same collection of imports and calls I do every time, and package it up in a Click entry point.
Inject basic CSS directly into the generated HTML. For now, just get the color scheme put into CSS variables and make sure I can use them in the output.
Get the SVG output a little more structured because I think I want to be using BEM to style everything, and stuff like "a hex" and "its label" should go in the same logical unit, right?

Coding 2024-09-16

2024-09-16T04:00:00-04:00

Here's what I'm seeing from slamming stuff together between a bunch of different libraries:

I need to write CSS to get things centered.
The svg viewport is cutting it off too soon at the bottom.
The svg is way too small.
All of the previous complaints I had about the svg still hold, because I haven't yet tried to do anything about them.

Hm. If I'm going to be writing CSS anyway, I should put a class on the svg elements and see if I can make that work.

That's probably going to need some tweaking, but I can make things work. For now, I'm going to research CSS methodologies. (Like, if you look under the hood on this blog, I'm doing something that's supposed to be my idea of BEM (although I'm apparently missing some aspects of it in the backend code), but maybe there's something else I'd prefer using for this project? I don't know.) Anyway, I should wrap up.

Good night.

Diary 2024-09-15

2024-09-15T04:00:00-04:00

I thought I was going to get stuff done today that I could blog about, but I ended up taking it easy and messing with things that I don't want to write up here.

Maybe I'll keep on taking things easy. We'll see.

Good night.

Coding 2024-09-14

2024-09-14T04:00:00-04:00

I didn't try to get too much done today. Here's what I got:

Installed dependencies to generate and process html.
Added MANIFEST.in file that should make things work if I ever package this up.
Basic attempt at a template.

What I need to try next is to write code to use the new template to generate a bare-bones HTML page, then convert it to PDF, and see how messed-up the result is.

I'll see about that tomorrow, maybe.

Good night.

Coding 2024-09-13

2024-09-13T04:00:00-04:00

All right, I want to read a book instead of stuff on my screen, so I'm just going to put down some quick thoughts.

I realized that I want the final output from this sector rendering process to be a PDF. So I need to figure out how to go from the SVG I generate to a PDF; it looks like there are libraries for converting from HTML to PDF, so I can embed the SVG in HTML and then convert it. This maybe means not saving it directly to a file, which is fine.

The big thing to look into is writing command-line interfaces, now that I have functions that do something useful when I call them over and over.

This all looks like reasonable stuff to work on when I'm a bit better rested. Speaking of which.

Good night.

Coding 2024-09-12

2024-09-12T04:00:00-04:00

Here's what I've got so far, from my attempts to generate a hex grid:

The hexes are black with a thin white border; they should be white with a somewhat thicker black border.
The viewport is too small.
The hexes are probably too big.
The hexes should be offset enough to provide decent margins.
I'm not even trying to render hex-specific information yet.

Other than that, it's nice to see idle planning generally come together. I'm still a little wiped out from how the last few days have been, so I'm going to get ready for bed now.

Good night.

Coding 2024-09-11

2024-09-11T04:00:00-04:00

Today was kind of rough, so let's just toss out some ideas, no pressure.

So, drawing hexes requires explicit coordinates for all points. Since I'm tiling them, any irrational numbers will end up compounded as the hexes continue sideways/down. Is that a problem? I don't truly know, but I would rather just use a small rational approximation to the square root of three, so I don't have to think about it. Seven fourths should be fine. I'm assuming that I'll end up wanting a "worse" approximation for aesthetic reasons of some kind, but seven fourths is a good number for not really thinking about it.

And speaking of not really thinking about it, I'm pretty tired already, so I'll just leave this here and start experimenting with it later.

Good night.

Weekly Roundup 2024-09-10

2024-09-10T04:00:00-04:00

Wednesday: I put together some plans for the code to help with Stars Without Number.
Thursday: I started designing a very complicated system for carrying out arbitrary rolls. I will not need to carry out arbitrary rolls any time soon.
Friday: I started trying to decide how to report intermediate steps in rolls to the user.
Saturday: I tried to use a ready-made third-party library, and got tripped up by the fact that it's ready-made for a subtly different use case.
Sunday: I decided to give up on adapting third-party code to this, and rewrite some old code from a few years ago.
Monday: I rewrote the code from a few years ago, and then, freed of the burden of not having those helper functions, tossed together deserialization logic thanks to cattrs, which is ready-made for pretty much exactly what I used it for here.

Next week, I will figure out how to programmatically vector-draw hexagons in a way that I'm happy with, among other things.

Coding 2024-09-09

2024-09-09T04:00:00-04:00

I've got enough of a stub for reporting to implement the helper code I currently want. (The code is missing nice stringification, and I don't want to handle that right now.)

With that done well enough for now, I tossed together cattrs-based deserialization for the sector data I generated previously.

I'm tired right now, but I think the next step is to work out image generation and potential route calculations. From there, I can work on figuring out how to represent actual routes in the data, and to deserialize that into some form of reasonable object. (NB self: I need to check whether having a working route in one direction automatically implies having a working route in the other direction.)

Anyway, that's enough for now.

Good night.

Coding 2024-09-08

2024-09-08T04:00:00-04:00

Well, I kept on reconsidering, and I'm still thinking the best course of action is to just dust off the designs I came up with years ago, and make them... somewhat less janky. Sometimes merely differently janky.

Now that I've committed to "okay, let's have this code in here", I do kind of have to ponder the choices I'm making for package layout. It's a little chaotic and thrown-together, currently.

Today was a rough day towards the end, so I don't have anything else to cover right now. Going to try to get some proper sleep.

Good night.

Coding 2024-09-07

2024-09-07T04:00:00-04:00

"Maybe I can just slot in some third-party logging library and it'll just do what I want."

...

Hm. Hm.

I'm... really not sure if I want to keep on going down this road. This is Eliot, which is meant for making it easier to analyze complex systems at scale. I'm trying to generate human-readable summaries of relatively simple workflows. Certain... tensions... between these goals are manifesting.

In light of that, I'm kind of leaning towards rolling my own system for dice logging, but maybe I'll reconsider by the next time I think about this.

Either way, it's late and I need to get to bed.

Good night.

Coding 2024-09-06

2024-09-06T04:00:00-04:00

All right, I've redone the rolling logic and gotten the interface roughed out. Now, I want to figure out how to report intermediate results to the user.

You've already over-engineered this code so much. Is there no chance that you can focus on the things that are actually relevant now, such as deserialization or image generation?

The missing capabilities irk me.

Now, to be clear, I've already got a previous attempt at this in my back pocket, and I could try to draw from that attempt, but I'd like to try for something consciously different to start with. (Not least because of some amazing conceptual contortions in the previous attempt.)

For now, I'll focus on a few questions of varying importance:

When I pop open a Python REPL and do a roll, where should the messages go?
If I have a pure CLI script, which output stream should the messages go to, and how should they be sent there?
How do I want to do all of this for some kind of TUI or GUI?

I need to ponder this some more, and look for prior art. For now, I'm going to get ready for bed.

Good night.

Coding 2024-09-05

2024-09-05T04:00:00-04:00

Well, I've worked out how to avoid template pattern stuff. The new design I'm working towards as a result is... a thing. There are a few bits that I'm missing and I need to sketch out how they should work.

Here's the idea of the new design:

There are a few "internal" dice types, which all accept a scaling factor. (I am not at all confident that Mypy will accept how I have implemented this.)
These are collected into a "pool", in a PBag, along with an integer offset.
The pool can be rolled, but this does not produce a final numerical result. Instead, it produces a mapping from the number rolled on each die, to a bag of the dice that rolled that number.
This should be sufficient for stuff like the obvious choice of summing everything up (simply scale by the length of the bag), as well as more involved means of scoring or determining an outcome, as well as allowing for intermediate processing steps like exploding dice or dropping or doubling extreme entries.

What I need to figure out now is the proper way to handle "pipelining" this stuff, as well as the "basic" case of "please just sum up these dice, crazy software design person".

I'll think about that overnight and see what I come up with.

Good night.

Coding 2024-09-04

2024-09-04T04:00:00-04:00

All right, I've tossed together some classes that should handle things. Let's review and update my goals from last time:

Get cattrs to successfully process the TOML file I wrote.
SVG output.
Review confusing planets
Stretch goal for now: overhaul my thrown-together dice implementation for superior testability and logging

The main thing I anticipate to be funky with cattrs is that I want to turn nested maps into a single map with tuple keys. This seems like it should be possible, but there's no reason for it to be out-of-the-box behavior. Every other part of the process should be straightforward

Concerns for the SVG output include:

How to balance ease of hex shaping with sensible coordinates
Applying a shear transform to the coordinates to compensate for what the X coordinate does to the Y coordinate

As far as the stretch goal, here's what I want to improve on...

Right now, each die simply calls random.randint to produce its results.
There is no indication that a die was rolled.
There is no indication of any die's results.
There is no indication of why a die was rolled.
I want nicer string representations for the different types of dice.

Tossing out some ideas for using context variables to address this:

The roll methods should take a reason argument.
There should be a callback that all roll methods call that records the reason.
The behavior of this callback should be customizable via a context variable.
The Dice compound type should suppress reason noting in sub-calls, and pass the reason through unchanged.
There should be a callback that all roll methods call that records the result.
This should be similarly customizable, but there doesn't need to be any normal code making use of the customization.
There should also be customization for the randomness source, in a similar fashion.

Some of this could be testable via mocking, but, eh, I'd rather put in the effort to make this work, than try to have mocking happen to work.

The other thing I need to consider is the level of template pattern usage I'm currently considering. It feels okay to me, but my not-quite-gut reaction is that I'm likely to reconsider this later.

I'll have to sleep on this and see what I think.

Good night.

Weekly Roundup 2024-09-03

2024-09-03T04:00:00-04:00

Wednesday: I tried to design a Click clone in Lua.
Thursday: I continued to try to design a Click clone in Lua.
Friday: I put down the Click clone stuff, and decided to look over what kind of rolls I need to deal with.
Saturday: I realized that it would be better to use Click than to try to clone it, so I made a game plan to replace "requiring a Lua module" with "reading a TOML file and then processing it a bunch".
Sunday: I filled in a bunch of stuff for the TOML file.
Monday: I filled in more of the TOML file.

Next week, I'm going to focus on properly reading in the data, so I can make nice visualizations of it.

Coding 2024-09-02

2024-09-02T04:00:00-04:00

The planet details are filled in, at least their initial values. There are a few that just seem kind of... weird. So, here's what I'm thinking:

Code up the options from this part of the book as a bunch of enums etc.
Figure out some means of SVG output.
Fill in more information about the planets that are confusing or relevant to my character concept.

I'm seeing a few options for svg writing, and I don't want to evaluate between then right now, because it's late and we did stuff today.

Good night.

Coding 2024-09-01

2024-09-01T04:00:00-04:00

I didn't touch any code. Instead, I spent some time filling in details about the inhabited worlds in the sector. I'm not done filling things in, and I'll probably need to reconcile a few details and possibly tweak some things to make sure that the topology of the sector is still right.

Once I make some progress, I'm going to try to get the deserialization code together, but first I want to have all of the fields present, if not properly filled-in.

Right now, I want to veg out and watch videos.

Good night.

Coding 2024-08-31

2024-08-31T04:00:00-04:00

Okay, I just did a bunch of questionable things, but I've made progress on the adventure background, so technically this is a gain.

Basically, I realized that the whole attempt to create Tap was a sign that I was trying to do things that were already solved problems. As such, I decided to go where the libraries I know are. Instead of tracking data in Lua files, I'm going to switch to Python, and try to rely on TOML. (This does mean that the data format doesn't have control flow, but, like, whatever.) I can then read it into a nested data structure, then use cattrs to convert it to attrs.

Those later steps shouldn't be necessary just yet. Not until I want to do stuff like convert the star sector data into a nice hex grid. Inside tech baseball: The capabilities of uv compile have me interested, even though it probably won't mean more for this project, and definitely won't mean anything until I specify some dependencies. Regardless, I'm maintaining a lockfile for this code, to see what happens. Hopefully nothing too exciting. I really do appreciate the modularity that uv has. Like, the virtualenv management? I don't vibe with that workflow right now. Not using it. uv pip is not judging me for this. uv pip is also taking commands that have nothing to do with pip's commands, but I have to imagine everyone is super-tired of that conversation.

Anyway, sleep.

Good night.

Coding 2024-08-30

2024-08-30T04:00:00-04:00

With all of that messing around yesterday, you'd think I'd be moving on to prototype Tap, right? Nope! Time to look at Mythic's rolls and remember how they relate to each other.

Rolls on the Fate Chart can invoke Random Events. Random Events first roll for Event Focus, which can involve the Lists; then, rolling for the Event Meaning. Starting a Scene requires a roll to determine the nature of the scene; this may then require pretty much any kind of roll.

Some meaning tables require an arbitrary number of rolls. Ultimately, it appears better to document how all of this fits together, than to try to fit it all into code.

Well, that was a little disappointing. I can take a look at the variations to see if anything catches my eye... Hmm, nothing I want to touch right now.

The quick summary of dice in Mythic is that I can get most things done with percentile dice, most of the rest done with a d10, and the remainder with a d4, d6, and d8.

In Stars Without Number, I see... d20 for saving throws, d6 for skill checks, d8 for initiative except I'm not going to bother rolling initiative, d20 to hit, d whatever to determine damage. There's a lot of stuff here.

...

I just took a quick look over the various things my character can buy when the time comes. Nothing has quite grabbed me yet.

At this point, I think the sensible course of action is to punt on the machine-readable ideas for now, print out and fill out a character sheet, and the other sheets, take pictures to keep track of things, and just start going.

Right now, though, bed.

Good night.

Coding 2024-08-29

2024-08-29T04:00:00-04:00

Well, I feel like doing a bit more planning, since there isn't really a point in writing elaborate subcommand infrastructure if it can't actually parse things for one command. I'm calling this project the Terminal Argument Parser; it's inspired by the Command Line Interface Creation Kit. So, let's look over the Click documentation and figure out what to copy and what to do differently.

First major thing: Lua does not care if you do ludicrous higher-order function stuff, but there's no syntactic sugar comparable to Python's decorators, so let's punt on the exact shape the interface takes.

Let's see what to keep from options. Definitely want to be able to specific different option names. However, dispatching values based on argument name isn't a thing in Lua, so the function that ultimately handles this stuff is just going to have to take a "namespace" argument and get values from it using option names. I think the exact same strings used on the command line should work. Options can have defaults, sure. And some form of converter function. Can mark it as required. That seems like it should be mutually exclusive with providing a default... If there is a default, can choose whether to print it in the help. Flag-type values need a bit more thought.

These don't take any arguments on the command line, and should toggle between two states, usually true and false. If I'm committing to not having a "destination" concept, then there are two possible situations: Either there's one flag argument that switches from one arbitrary value to another, or there are paired flag arguments that have opposite boolean values. I feel a slight aesthetic affinity for the latter, but I don't have a sense of what this project is going to actually need, if anything.

Multi value options. Not compatible with flags. Aesthetically, they make me want to make "vanilla" converter functions "pervasive", applying the same mapping operation to every argument; specialized multi-value converters can then only work on specific numbers of arguments. Note that doing it like this would not set nargs, so perhaps there's an argument to be made for dispatching on whether the converter is passed as a function or an array. (Of course, it could be a functable, so maybe a table with a metatable should be treated as a function?)

Multiple options. I may have put myself into a bit of a corner. Here's what makes sense to me: If multiple options are supported, then that's handled by multiple returns. If multi values are supported, then the result of converting is somehow collected into a single table. So if you do both, then you get a bunch of tables.

Counting. Is a type of flag. I assume it could have a default value of some integer, but I'm not sure what that would accomplish that couldn't be done by just defaulting to zero and adding a number later.

Boolean flags goes over stuff I mentioned earlier...

Then we've got feature switches. By the reasoning I had above, Tap would have this as a set of flag values in which no more than one can be true at once.

Choice options... are a type of converter.

Prompting is another dimension of variation. I don't know how much of a pain it is to support the password-related features.

I guess stuff like show_default, eager parameters, and callbacks need to be a thing... Woof. Eager parameters imply, I think, that the stack of subcommands needs to be built up ahead of time. Which is doable, just, kind of a pain. No need to worry about an equivalent to expose_value I think; if you don't look for it, you won't see it.

Want to implement a yes parameter, then write a helper for it...

For environment variable stuff, the prefix has to be passed to the top-level call, and choosing an environment variable requires that Tap does choose a canonical name. I'm not so interested in automatically generating an environment variable name, as I am in specifying a static one, so I can do something like export CHAOS_FACTOR=5. That may be simply a dynamic default.

Skimming over arguments... These either need to be specified in a clear order via sequencing of calls, or simply passed in a list. Aside from that, most of the work to support arguments appears to be in custom converters.

I won't say all of this is simple—it frankly seems like a huge confusing tangle—but I've got a starting point for drawing a confusing diagram of how different options relate to each other, and that's really all I wanted for now. Anyway, I need to get ready for bed like five minutes ago.

Good night.

Coding 2024-08-28

2024-08-28T04:00:00-04:00

Okay, I've tried to work out some stuff for this Lua code in my head, and it's not working out. Let's try to hash this out in a blog post.

I want to have Lua modules, which can be used in, technically, three different ways:

Object which can be used in other code.
Command-line endpoint that carries out some action.
Subcommand that some other command delegates to.

I've managed to write a utility function to check whether some code is running as a direct result of a lua command, via if debug.getinfo(4) then. It's weird-looking and unintuitive, but it works.

Now, let's think about command-line parsing. I want the result of parsing to be a table, which is passed to a function...

Thinking about stuff like cleanup and control flow, I think this function should use the coroutine library to do something sort of like Python's context managers.

So, let's see how I think things should go together. There should be a function that takes a table input, and performs some action. There should be a function that converts a list of strings to a table.

This table may contain an array of unparsed strings, and a similar pair of functions.

These need to be handled by a runner that takes a table containing those functions, and optionally a list of strings.

Then, the debug.getinfo wrapper needs to be able to take a callback for the script case.

All of this has, of course, completely ignored the logic behind "parsing out arguments and options into a table".

Here's a weird idea: instead of collecting option values into a table, return them from a function, so it just uses multiple returns if there are multiple values... This idea would have a slight issue of not distinguishing "didn't pass anything to a flag that takes an optional argument" from "didn't pass the flag at all"... At least, it can have that issue, depending on the details of how I implement it...

Wait, so, you're trying to represent RPG character sheets in a machine-trackable format, right?

Yes...

:) (glitchier)

So why the HECK are you trying to clone Click in Lua‽

Haha, that's all for now folks, I'll try to start coding stuff up tomorrow.

Good night.

Weekly Roundup 2024-08-27

2024-08-27T04:00:00-04:00

Wednesday: Decided to play a tabletop RPG. Not sure why I ended up shaving that yak.
Thursday: I will be trying to describe actual details of Stars Without Number, since it has a pretty generous free version.
Friday: I realized that trying to automate the "referee" aspect of GMing Stars Without Number would probably not be worth it.
Saturday: I realized I should actually interact with the rules, and thought about the right way to represent dice.
Sunday: A little more character creation...
Monday: A little more character creation, but soon it will be time to code more.

Next week, I'm going to be futzing around with stuff in Lua, and getting my character a bit more locked in.

Coding 2024-08-26

2024-08-26T04:00:00-04:00

Background in, starting skills in, currently missing a huge amount of scaffolding code, and psyching myself up to choose a character class. It's basically a question between, do I try to synergize directly with my backstory, do I put together a mildly amusing concept that will probably expose me to more complex rules, or do I try to offset the lethality of concept by specifically specializing for it, meaning that I'll kind of, you know, need to get into combat to take advantage of the build, or do I try to thread the needle between two of those options? I'm leaning towards mildly amusing, but may I should look over the psionics rules and see if that changes my mind.

...

Okay, fingers crossed, I think I can handle this.

Hopefully, I don't eat those words.

Anyway, I really need to get ready for bed.

Good night.

Coding 2024-08-25

2024-08-25T04:00:00-04:00

Well, I've got a bit more done. I've picked out the background for my character, and decided on my starting skills, so next I just need to figure out how to code those in.

I have decided how I should handle die rolls, but I don't want to touch the code right now; I'm pretty tired and I just want to zone out.

I'll get on that now.

Good night.

Coding 2024-08-24

2024-08-24T04:00:00-04:00

Well, I'm never quite working on what I expect to, but I'm just going to keep noodling around for a while. I thought to myself that I'd like to have something very simple for handling command-line interfaces, and I've just kind of tossed a bunch of modules together that should let me do that.

At this point, I think what it makes sense to do is to actually build a character, and start assembling the character sheet library around that. Once I'm ready to actually get started, I'll need to put together a few things on the Mythic side of this all.

The big thing I'm trying to work out right now is how I want to handle die rolls. Like, does renaming the math.random function get me anything, and, if so, how do I want to go about it? Do I want each kind of die roll to just be a nullary function, or should I try to do stuff with metatables so it's like, the number of sides is foregrounded, and I can make the whole thing more data-y, which is definitely a word you should use in a serious presentation.

Something like

local fudge = die(6, {-1, -1, 0, 0, 1, 1})

Hm. Not quite vibing with this for some reason. Going to have to take some time to think about it. For now, bed.

Good night.

Coding 2024-08-23

2024-08-23T04:00:00-04:00

I looked over the combat rules for Stars Without Number, and, oof. The bookkeeping. I can't automate this off-the-bat.

Here's my current plan:

Go ahead with writing a character sheet in Lua, including nice output of some kind.
Write up references for all applicable rolls and actions.

Maybe I'll have more to say once I get those done, but for now, I'm just cooling off and doing random other things. Got to go for now.

Good night.

Coding 2024-08-22

2024-08-22T04:00:00-04:00

I got LSP integration working, and it informed me that some of the utility code I wrote was incorrect. I addressed that.

Anyway, I was going over the rules I'll be using, to make sure there aren't any major incompatibilities, and it all looks fine I think.

In the past, I've felt weird about documenting stuff about paid RPG rules, buuut this is Stars Without Number, which has a free version that I believe covers all of the cogent stuff for, like, basic character interactions. (Like, I'm not expecting to dive into the deluxe-exclusive stuff right off the bat, but it's there if I get into a Situation.)

I should be ready now to start implementing Stars Without Number statistics stuff. But for now, bed.

Good night.

Coding 2024-08-21

2024-08-21T04:00:00-04:00

I decided to shift gears from complicated Python code and writing focused on worldbuilding to try out solo roleplaying with minimal worldbuilding.

And how's that going?

Well, I want to put my character sheet and transcript in revision control, and write helpful code to handle stuff like rolls and attribute bonuses. So... currently I'm trying to debug why Kakoune doesn't seem to be starting lua-language-server.

Hoo boy.

I'll try to get this all squared away tomorrow. Anyway, sleep.

Good night.

Weekly Roundup 2024-08-20

2024-08-20T04:00:00-04:00

Wednesday: I switched Structured Data's testing to pytest.
Thursday: A little more work on Structured Data.
Friday: Nothing much.
Saturday: Switching off of Ward was kind of a bummer. I wonder what it would take to clone it using techniques from MOTR.
Sunday: I reviewed some of my old notes.
Monday: A little more work, but I think I need to take a different approach somehow.

Next week, maybe this, maybe something else...

Coding 2024-08-19

2024-08-19T04:00:00-04:00

Hm. I really would like to get some progress made on MOTR, but it's slow going to review my notes. I am putting in some work on other things, so that's something. I just need to work out some way to write up those things instead of "incremental progress remembering how MOTR should work".

I don't have anything else for tonight, so I'm going to see about getting ready for bed.

Good night.

Coding 2024-08-18

2024-08-18T04:00:00-04:00

My focus today ended up mostly on the kind of stuff I don't blog about here, so there's not much to say right now, but I did look over my notes for what improvements I need to make. I think what I should do when I'm ready is to print some of those notes out and work with them as physical objects for a bit.

Basically, I need to take ideas that I came up with weeks apart, and put them together into a checklist. I need to get over the hump into actually doing something with that, so I'll get the major notes printed out and get ready for bed, then fill in some of the other ideas later, hopefully tomorrow.

Good night.

Coding 2024-08-17

2024-08-17T04:00:00-04:00

Hm. Today, I ended up putting some stuff together for a Ward-inspired test runner. However, one of my other inspirations was the Matrix system from MOTR, so if I want to get anywhere with this, I should probably finally revisit MOTR.

I'm going to have to take a bit more time to recover from this week, but maybe I'll get some inspiration. For now though, getting ready for bed.

Good night.

Diary 2024-08-16

2024-08-16T04:00:00-04:00

Okay, I'm out of it today, and I'll try to get something interesting done tomorrow. No promises.

Good night.

Coding 2024-08-15

2024-08-15T04:00:00-04:00

Okay, I did a few more tweaks to Structured Data's code and linting. I don't think there are any more small changes to make; I either need to switch projects or make actual progress here.

THere's a chance I would have gotten more done today, but I decided to install some software on my laptop, and that was perhaps a foolish endeavor.

Maybe I'll get more done tomorrow, or maybe I'll try taking things easy. Either way, it's already late.

Good night.

Coding 2024-08-14

2024-08-14T04:00:00-04:00

A bit over a month ago...

"I've got so much experience with Pytest, I'd like to give something new a shot. Like Ward."

Unbeknownst to me, literally the next day...

"I am no longer actively maintaining this project."

...

So.

I guess I should port it off? This is, like, slightly inconvenient in some ways, and slightly convenient in others, so, ultimately, who can say how I should feel about this?

...

Anyway, I switched to Pytest. Once I got the lints to pass, everything worked, so I'm confident that I didn't break anything.

Next thing to think about is how I want to try to handle deal, now that I have a runner that has proper hypothesis support. I'm not going to focus on that right now, though. I need to get to bed.

Good night.

Weekly Roundup 2024-08-13

2024-08-13T04:00:00-04:00

Wednesday: I put together some general ideas for Open Divination. (I have not checked whether that name is taken. Whatever.)
Thursday: I reviewed the Rust tutorial, and mostly did what I was supposed to for it.
Friday: I discovered that the default wasm-pack template appears to not be what I want for the project layout I have in mind.
Saturday: I got things more-or-less working in a different way, so that seemed promising.
Sunday: I researched browser extensions, and determined that I need a manifest.json file, but did not determine whether I want to be writing it by hand or generating it somehow.
Monday: Oops, ended up improving the error reporting out of Structured Data instead of any of that.

Next week, I'll probably work on, you know, something.

Coding 2024-08-12

2024-08-12T04:00:00-04:00

My attention switched tracks, so I ended up fixing up some of my Structured Data code to look... not horrifying, mainly by shoving a bunch of it into a Jinja template.

Then I went looking for mutation testing for Jinja templates, because I'm sure I remembered that being some kind of thing, and the results were... frustrating.

:) (glitchy)

Oy, Google. Don't give us results for Ethereum, of all things.

Anyway, I forget where I was going with Structured Data, so either I'm going to remember what I'm doing, or get back to understanding browser extension packaging. For now, I need to get ready for bed.

Good night.

Coding 2024-08-11

2024-08-11T04:00:00-04:00

I'm still trying to get a handle on how best to do extension stuff. Like, okay, I need a manifest.json file. Is that something that it makes more sense to generate or hardcode? Where is the extension-writing guide for people who got turned off of JavaScript like ten years ago because "Uh... something about variable scoping"?

I know I need to do a lot more research here, because otherwise I'll put together something that only mostly sort of works, if that.

Thinking about what I'm trying to proof-of-concept here... I have a common library. It is, I think, getting bundled into a wasm file in a pkg directory. I believe I want every file in that directory to be somehow accessible from a directory with a manifest.json file. On top of that, the manifest.json file has to somehow point at those files so that they actually get used.

I'm not going to figure out anything else right now; I've got to get some sleep.

Good night.

Coding 2024-08-10

2024-08-10T04:00:00-04:00

Okay, I did a little work on this today. Basically, I redid the WASM stuff sort of from scratch, and now the configuration makes more sense to me. I've got stuff going on that's building, and either it's horrendously broken, or it's basically working perfectly.

The next step is to figure out which of those possibilities is the correct one. If it's the latter, then I'm going to see about integrating the built WASM into a browser extension. Basically, I'm trusting that I'll be able to write the actual code when the time comes, so I'm focusing on the stuff that I know the least about.

Like, sure, I can apparently write functions that get exported through WASM. Now, what do I need to write to make a web extension work? If I can get this all the way to the end, then I can start writing code, but if I hit a hurdle of some kind, I'd like to know now so I can figure out what I need to do differently.

Anyway, I have been very tired and I desperately need to sleep, so I'm going to cut this off now.

Good night.

Coding 2024-08-09

2024-08-09T04:00:00-04:00

"Okay, let's make a virtual manifest."

...

"And a common library crate."

...

"And a binary cli crate."

...

"And a wasm-pack crate for the browser extension."

...

"That last one has me feeling like someone just walked over my grave."

So, I've got some stuff to read over to figure out how and whether I should even be doing some of this. None of that is happening right now. It is late, and I am incredibly tired because of reasons.

Good night.

Coding 2024-08-08

2024-08-08T04:00:00-04:00

Okay. I tired myself out severely today, so I decided to just review the basics of Rust. I even followed the directions!

All of the directions?

Look, doing all of the stuff they said, and a few extra steps, is still following the directions.

Uh-huh.

Anyway, I need to get to the point of more advanced configurations, so I can put together multi-crate stuff.

For now, I need to get ready for bed.

Good night.

Coding 2024-08-07

2024-08-07T04:00:00-04:00

Okay, let's switch gears a bit. One of the projects I've been thinking about some lately, is trying to compete with LLM assistants by providing easy access to traditional, algorithmically efficient divination techniques. This is not a total non-sequitur; one of the stated use cases for LLMs is "bouncing ideas off of", and I don't see that, for such a use case, "the tool has no understanding of the question" should pose any more of a barrier for traditional divination than it does for LLMs.

I've put together a basic outline of the kind of functionality I want to have, and I'm trying to figure out what's the right way for me to deploy this as many kinds of "app". Like, it simply makes sense to me that this should be some kind of command-line tool, daemon server, and browser extension. The obvious language for writing a browser extension is JavaScript, but, like, I dunno, I don't feel like using JavaScript for anything other than cheating at idle games. From some research, it seems like it should be possible to target WASM, which is possible with Rust, which I'm somewhat familiar with. Although, I also remember hearing about unpleasant experiences trying to target WASM and non-WASM at the same time, but maybe the precise details of that don't apply to what I was thinking of, which was to have completely separate projects for command-line versus browser extensions.

The basic thing I need to figure out, however I decide to swing this, is, how can I notice as quickly as possible that a given approach will not work?

Core logic in library
Binary crate, probably using clap
Build extension with wasm-pack, and probably other things

I'll have to try these out later, but I'll get ready for bed now.

Good night.

Weekly Roundup 2024-08-06

2024-08-06T04:00:00-04:00

Wednesday: I went through a few different Python documentation linters.
Thursday: I read and thought.
Friday: I made some plans to beef up the core interfaces in the new Structured Data.
Saturday: I took things easy.
Sunday: I wrote some code that my syntax highlighter chokes on, but Ruff doesn't complain about.
Monday: ȾĦɆ ꞦȺꝞȺ₲ɆꞨ ꝊӺ ȾĪᛗɆ ɃꞦɆȺҞ ĐꝊⱲꞤ ȺꞤꝊȾĦɆꞦ ȻꝊꞤꞨȾꞦɄȻȾ ꝊӺ ꞨĪŁĪȻꝊꞤ ȺꞤĐ ꞨꝊŁĐɆꞦ

And then I tried my best to take things easy.

Next week, I'm going to find something satisfying to work on.

Coding 2024-07-28

2024-07-28T04:00:00-04:00

Okay, the code is updated to produce error reports when a match doubles up on bindings. The only problem with the code as it stands is that it's kind of an abomination. It's a triply-nested multi-line f-string, so I kind of have to try to come up with something better.

Aside from that, the next thing I can do is either to implement more classes from the old version, or get back to trying to set up the documentation. Neither really appeals to me, so maybe I'll try to change gears soon. I'll have to take some time to think about that. For now, I should get ready for bed.

Good night.

Diary 2024-07-27

2024-07-27T04:00:00-04:00

This week was a bit rough on me, so I did my best to take things easy today.

I can't see that I have anything else to think about tonight, so I'm not going to force it.

Good night.

Coding 2024-07-26

2024-07-26T04:00:00-04:00

I made some minor improvements to the Structured Data code today, but mostly I was focused on planning. See, one unimplemented feature is that it should be impossible to try to bind the same name twice or more in a single call to match(). Now, it's not too hard for me to get that working, but really, I should be analyzing the structure being passed in well enough to point out where it's going wrong, and getting that right takes a bit more thought.

Idle speculation:

Currently, I'm having the iterator that produces this stuff yield item-like pairs. If I change from passing those into a dict constructor directly to building up a dictionary, then I can add additional fields to the iteration (at some point it makes sense to create a class for holding this stuff, but let's not think about that yet). I can include a field that indicates the "path" to the current name; this should be a tuple of structure field-names, probably. None of this should be too hard, but I'll have to consider whether there's something I'd rather do.

I've let things get late, so I won't try to do any more tonight.

Good night.

Diary 2024-07-25

2024-07-25T04:00:00-04:00

Okay, so. I was thinking about Structured Data today, but mostly I was reading and thinking about writing. I need some time to think about these ideas, and I should take my time and just get to bed now, because I was reading for a while.

Good night.

Coding 2024-07-24

2024-07-24T04:00:00-04:00

I've just gone through several different tools for checking docstrings, and settled on what appears to be the least bad one, although it is... not great. I feel like if a tool is going to try to tell me whether I've used an iterator type in a return annotation, "evaluating type aliases" should be table stakes. I'm not expanding them! The full annotations are disasters, and I need the aliases to make it possible to write these annotations, let alone read them.

Anyway, I did manage to improve the documentation a bunch, I think. In order to tell, I need to finish setting up Sphinx, which never feels quite natural to me in the context of documenting Python code, the thing it was designed to do.

I think I'll try to work on test coverage for a bit, but to do that, I need to make some decisions about code behavior, and I'll have to sleep on some of the relevant questions there. It's already too late right now, so I'm going to wrap up as soon as I can and get to bed.

Good night.

Weekly Roundup 2024-07-23

2024-07-23T04:00:00-04:00

Wednesday: I thought about splitting up the Structured Data code.
Thursday: I worked on increasing the coverage.
Friday: I made some improvements, or at least changes, to the code structure.
Saturday: I found out that parso, and therefore everything that uses it, doesn't work with PEP 695 code. However, I did find a mutation testing tool that doesn't use parso. It did not work, to start with.
Sunday: I got it working, and then got it sort of not working.
Monday: I fixed the problems I was having, and started thinking about improvements to make besides increasing the mutation coverage.

Next week, I'm going to keep on improving this code, but also to try to scrape together the focus for writing.

Coding 2024-07-22

2024-07-22T04:00:00-04:00

The revelation that fixed my mutation testing: "Oh yeah, if a bunch of things are touching the same cache file at the same time, they usually assume that's not what they're doing. And a crash gets treated like a test failure."

So now my mutation testing is slower (~7 mutants per second) but it's more reliable.

I've also implemented a bit more functionality, which helped to write tests that caught a few more mutants.

Rather than trying to grind out the final remaining mutant, I think for now I should try to write subtler tests that catch cases that I've been glossing over in the name of getting stuff implemented. Stuff like "some pattern names should be rejected" and "some additional binding names should be rejected" and "there's all sorts of extra functionality that's currently not implemented because I just haven't written tests for it yet" and "it should be an error if the same name gets bound twice during a match".

That's a bunch of stuff to handle, but for now, I'm going to be happy I made progress, and get to bed.

Good night.

Coding 2024-07-21

2024-07-21T04:00:00-04:00

Everything is working much better now. Last night, I managed to go from doing one thing wrong, to two things wrong, so then, every time I fixed one thing, it didn't help, so I put it back. Oops.

Because poodle (1) wasn't providing information about my weird project layout (2) in a format that Ward (3) could readily use, I had to put together a little shim to invoke Ward and Mypy properly. Once I had that, it was obvious that I'd messed up the tox configuration for poodle, so I had to fix that, and then I finally had it running mutation tests.

From there, I was able to improve the mutation coverage. It's not all the way up yet, but I'd like to split more of the code out into new modules before I try to mop up the stragglers.

...

Okay, I did that, and it had the somewhat unexpected consequence of making some of the mutants sometimes get detected? Something's a little wrong there, but I'm not sure what. I'll work on it later. For now, reading and bed.

Good night.

Coding 2024-07-20

2024-07-20T04:00:00-04:00

I got mutmut all set up, then found out that it doesn't work on PEP 695 code. Actually, I don't think I can expect any mutation tool using parso to work with this code.

I just found one that looks to use the builtin ast module, so, like, that had better work...

...

It's not, but not for the reason I expected. I'll have to figure out what's breaking test discovery in the morning. Anyway, I'm going to go lie down.

Good night.

Coding 2024-07-19

2024-07-19T04:00:00-04:00

Once I got tox passing yesterday, I worked today on Weird Tricks to cut down on dependencies between classes.

At this point, I would be all set to start separating out classes into modules, except that my eyes are screaming at me to stop looking at screens, so I'm... I'm going to just have this be a short entry and go lie down.

Good night.

Coding 2024-07-18

2024-07-18T04:00:00-04:00

Okay, the next thing I wanted to do was to get all of my tox environments to pass. (Insert ramble about how the way MOTR does it obviously makes so much more sense.) To get coverage to full, I need to write tests for the following cases:

Creating an AbstractEnum class with no members.
Error case: creating an AbstractEnum class with members and abstract methods.
Creating an AbstractEnum class with members and no abstract methods?
Iterating over a MatchDict.

That last one was easy, so I did it. Let's see about the others...

Hm, the question mark one didn't do what I hoped it would. Let me see...

Ah, I was able to cheat that branch out. Now I just need to address the warnings and errors from Ruff.

I'll work on that while this publishes.

Good night.

Coding 2024-07-17

2024-07-17T04:00:00-04:00

Okay, it's a little late, but I can still swing this...

I'm looking over the code I have written so far, and trying to decide if I want to break it down. The obvious candidate for putting stuff in its own module is MatchDict. Like, all any of the rest of the code really needs to know is that it's a mapping with a custom constructor. The less obvious candidate is the stuff related to the Structure type. I feel I should hold off on that until I've worked out the proper custom constructor stuff for Patterns, and how that interacts with Discard. Lastly, there's this compatibility class I, um,

Copied and pasted from StackOverflow

Right, that... It handles the metaclass differences between enums and abstract base classes.

Anyway, while these all seem like basically solid bets, I guess what it really makes sense to do is to sketch out the new functionality I want this code to have...

Helper function to convert non-scalar structures to CompoundStructures.
Context variables and context managers to allow temporarily registering additional converter functions.
Elaborate __getitem__ logic for MatchDict.
Shortcuts for creating Patterns.
Other CompoundStructures.

Ultimately, the big thing I need for reorganizing is to make a constructor for Patterns, which should lock down all of the important relationships.

Okay, I've got that done. I think what I'd like to do is get tox passing, and then start splitting things up.

Anyway, sleep.

Good night.

Weekly Roundup 2024-07-16

2024-07-16T04:00:00-04:00

Wednesday: I did planning for testing Structured Data.
Thursday: Corny old-timey sci-fi from the forties and fifties.
Friday: A little bit of work on Structured Data.
Saturday: A little bit more work and planning.
Sunday: A bunch more planning.
Monday: I just bit the bullet and cheated some code into existence.

Next week, I'm going to try to recover from some coverage hits I racked up earlier today, and plan some writing.

Coding 2024-07-15

2024-07-15T04:00:00-04:00

Well, I skipped ahead a bit in handling the advanced destructuring cases for matching. There are a few mitigating factors:

Coverage did not decrease
I only implemented the functionality required to pass the current test suite
I didn't abstract things out as far as in the previous version

This should provide a good basis to work from as I start testing more capabilities. I'll have to try that out later. Right now, I need to get ready for bed.

Good night.

Coding 2024-07-14

2024-07-14T04:00:00-04:00

I poked some more at the code today. One improvement I made was to change the type signature for advanced destructuring to one that doesn't require me to do elaborate mental gymnastics to verify it. This will remove some of the stranger lines present in the previous version, so that's a win.

Now it's time to think about how I might implement handling advanced destructuring. The "obvious" solution is to recursively match the substructures and then perform a merge operation on the outputs. A possible refinement to such an idea is to try to transform that recursion to iteration. The logical(?) endpoint(?) of this is the system I had in the previous version, in which advanced matches produce some amount of "additional work" that is put on a stack. (I seem to recall coming up with some very convincing theoretical reason why a stack is preferable to a queue, and it has nothing to do with "efficient stacks are slightly easier to create in Python than efficient queues".)

I think at some point I'm just going to "cheat" and skip to implementing most of this logic, and screw whatever this means for TDD purity. I should at least sketch it out and see if I run into any awkward areas first, though.

Anyway, I would very much like to get to bed now.

Good night.

Coding 2024-07-13

2024-07-13T04:00:00-04:00

I wrote another test, and got it passing, and now I'm fighting with my text editor configuration.

...

Okay, I think I "fixed" it.

The test I added was for comparing matching scalar values. The next test to add is for comparing different scalar values.

After that, oof. I'm still not really sure how to make the leap there. Even if I specifically only test using the discard value, which is the simplest structure to implement, it sure looks to me like my only choice is to implement a large amount of machinery to get that working. (I guess one thought is that I can hard-code that it special-cases to tuples to start with. That sort of helps...)

That's what I'm going to have to ponder to get this working. For now, sleep.

Good night.

Coding 2024-07-12

2024-07-12T04:00:00-04:00

I determined that I'm not yet ready to write contracts for this code, so I put together some basic tests. They're not especially rigorous from a TDD perspective, but they're working. I've already implemented this stuff once, though, and I'll be fascinated to see if I can find a path from "an implementation that doesn't handle non-scalar values" to "an implementation that does handle scalar values".

I'll have to give this some thought; not right now though.

Good night.

Diary 2024-07-11

2024-07-11T04:00:00-04:00

I've been reading a bunch, and trying to decide what I think. It's mostly been the Lensman books. They're historically significant (they influenced a lot of major space opera series), and are very good at being page-turner melodramas, and I really want to get a handle on some of the writing techniques on display, but there are some amazing blind spots in some of the attempts at being enlightened.

I don't want to be seen as trying to dunk on E. E. Smith, but he's fifties-ing hard in his writing, and I don't think the direction it's going at the end feels like an effective course correction; I'm not at all sure it was supposed to be. To be honest, although I've seen plenty of cheesy science fiction movies from the era, I'm not eager to go one way or the other on "standards of the time". Like, some of this stuff seems like it should be so obvious.

Anyway, I'll ponder that for a bit while I get to bed, because it's late again.

Good night.

Coding 2024-07-10

2024-07-10T04:00:00-04:00

I'm not quite comfortable writing code yet, so let's take a look at the kinds of contracts I want to have.

The match function takes a Structure argument; this type is a union of many types. The straightforward stuff is:

Discard should match any input, and result in an empty MatchDict object.
Pattern should match any scalar input, and result in a MatchDict with one entry, keyed to the name of the pattern.
A scalar structure should match an input equal to itself, producing an empty MatchDict, while any unequal input should fail the match, resulting in None.

The minor complicating factor is that I don't remember what I think matching pattern to pattern should do.

The major complicating factors are around, for one, "complex" types like tuples; there's a large amount of scaffolding around treating them in a sophisticated fashion, in the old code, and I can't jump into that. And the other is what I'm currently calling CompoundStructure. This is a marker/abstract class, and I can't see any way to express all of the expected behavior at the match function, where it's relying on polymorphism to get the behavior. Ultimately, these are kind of the same thing, it's just that I can't force builtin types to subclass from my marker.

With that in mind, it's possible to write contracts for the easy cases above, and to suppose that the hard cases should delegate to the custom class. Furthermore, those cases should probably have separate... cases...

I'm not sure whether Deal can handle the kind of type annotations I want to throw at it. I guess I'll just have to try it.

Anyway, it's late and I want to get ready for bed.

Good night.

Weekly Roundup 2024-07-09

2024-07-09T04:00:00-04:00

Wednesday: I officially picked Structured Data back up.
Thursday: I did some planning, and made a few decisions that I knew full well would annoy me later.
Friday: I ran right up to the edge of "later" by deciding to run Coverage with tox.
Saturday: So, this is "later", and I got Coverage working. ... So now I want to think about mutmut...
Sunday: I backed myself into something that maybe sounds like TDD, but, eh.
Monday: I started planning the use of yet another tool.

Next week, I'm going to see if Ward and Deal can at least sort of work together.

Coding 2024-07-08

2024-07-08T04:00:00-04:00

My plan is to try out using Deal to help write tests, so I've been thinking some about the basic assertions I'll want to have for the Structured Data interface. Some of the ideas I've got so far are kind of a pain to deal with. Like, what am I supposed to do, as far as writing tests goes, to deal with "there are three basically different things this function is supposed to do, and which one obtains is controlled by the precise relationship between the two inputs"? It's possible that I'll need to accept more work initially, and hopefully I'll get nicer stuff down the line.

Right now, I think what I want to do is figure out whether to try to disable Deal outside of the optimized case.

I'll just let that idea bounce around, and maybe start writing stubs. For now, it's late and I should get to bed.

Good night.

Coding 2024-07-07

2024-07-07T04:00:00-04:00

I refined the tooling a little more today, and did some planning. I think at this point, what makes sense to do is to start putting together a specification, and then devising tests to match it.

I know it's been a while since I did anything resembling TDD, but I've effectively got a greenfield project here, so I've already cranked up the thresholds, and it seems to me that something like red-green-refactor is the easiest way to deal with the thresholds being at maximum.

It'll probably be a few days at least before I'm ready to try actually writing code, but I'll try to write specifications or xfailed tests, or something.

Anyway, I let this get too late and I should get to bed.

Good night.

Coding 2024-07-06

2024-07-06T04:00:00-04:00

All right, I got Coverage working on this project. The only real hiccup was that I ended up having to switch from one particular configuration key to another, because of a reason.

The limitations I'm perceiving with the way that I got Coveragy working with tox have strengthened my resolve to get the Coverage wrapper written for MOTR, but right now, I'm focusing on planning for that rather than writing code.

But right now right now, I want to focus on the Structured Data rework. Right now, I've got a weird scattering of attempts at updated code in a single file, and I would like to start breaking it up some. Which means: more planning.

Actually, I guess I've got one more tool I want to try setting up: mutmut. I'll try and get that set up once the test coverage is, like, a little higher. To get that to work, I may need to figure out some stuff with the runner logic, but hopefully not.

Anyway, I'll get into planning one or both of those things tomorrow.

Good night.

Coding 2024-07-05

2024-07-05T04:00:00-04:00

Oh man, future me is going to hate the state that I've left this code in.

Aside from that, I've been working on getting tooling put together for this project. I've got almost everything I want put together, but I'm not going to be satisfied until I start measuring test coverage again. I'm well familiar with the kind of ceremony required to make me happy with Coverage.py, so I'm going to try something a little diff— Never mind. The different coverage tool I tried using managed to collect nothing, and I don't see any documentation, so let's psych ourselves up to push tox in the exact direction that got me looking into alternatives. Yay.

Well, I won't make myself feel any better about this by staying up longer.

Good night.

Coding 2024-07-04

2024-07-04T04:00:00-04:00

I've been focused on planning and research for this revival. Like, the whole "fractal tree" structure, to me, points to trying something like Ward as opposed to pytest, and also it should work better with Deal than the setup that I, personally was using in previous projects. Because MOTR is not ready, I'm going to try to just toss stuff together with tox, and see how far I can get with that, and part of that is to figure out how to fit it all together.

Right now, I'm trying to decide whether to keep the name the same and embrace a bunch of compatibility breaks, or just leave the abandoned library be, and design a new one.

I'll let those ideas percolate for a bit; it's late and I'm sleepy.

Good night.

Coding 2024-07-03

2024-07-03T04:00:00-04:00

Oh yeah, I'm bringing this back.

Oh hey, what's going on in h—Good lord.

That's right. The project that I shelved because I kept on going to or past the cutting edge of what Python makes possible. As I recall, when last we left off, I'd removed some code that attempted to emulate the capabilities provided by PEP 570.

And, well, we're so much further along, years down the line. I'm particularly excited to see what PEP 614 will allow for Structured Data. Or, at least, part of it.

See, Structured Data more-or-less broke down into two parts:

Some Haskell-inspired custom class stuff in the genre of "like attrs, but not attrs"
A Haskell-inspired matcher system that I'm pretty sure is capable of things that can't be syntactically expressed with the (comparatively) new match statement

I recently took another look at the code, and I was seriously torn between "this is convoluted and absurd" and "but this is really cool actually". So, why not take a fresh start, with my years of experience, and, um, actually understanding the Generator type, wow.

Anyway, one other thing occurred to me to try. Every once in a while, I experiment with a somewhat out-there idea. Sometimes, the result is pain. Just pain. So much pain. But I want to be open to new things. So, I was poking around, and I stumbled across a blog post describing how the authors laid out some JavaScript projects. I figured I'd try adapting the ideas to a Python project, and see what happens if I try slotting the Structured Data matcher rewrite into that framework.

To start with, I'm going to do a bunch of planning, and I've done some of that today, and I'll do some more of it later. For now, bed.

Good night.

Weekly Roundup 2024-07-02

2024-07-02T04:00:00-04:00

Wednesday: I tried to make the Dafny code less brittle, and it became more brittle.
Thursday: I investigated the "real" brittleness, and concluded that addressing it would require some kind of methodical enumeration or intuitive leap, neither of which I felt like trying to do.
Friday: I ran out of steam.
Saturday: I continued to be out of steam.
Sunday: I switched tracks to some code I abandoned years ago.
Monday: I realized that I need to take a break.

Next week, I'm going to try to take a break. It's not going super well so far.

Diary 2024-07-01

2024-07-01T04:00:00-04:00

I'm messing around with various things, but still not doing enough work on anything to think "Yes, I have a blog post I can write about this."

It seems I should be actually trying to take a break... Perhaps I should be playing some of the games I just bought on sale...

Anyway, I won't do any better if I stay up any longer.

Good night.

Diary 2024-06-30

2024-06-30T04:00:00-04:00

I haven't come up with any sudden revelations for how to improve the Dafny code. Maybe I should focus on other areas for now, and come back to improving the brittleness (or weakening the bounds) later.

Instead of bouncing off the Dafny code, I've been refreshing my memory about some code that I wrote several years ago and then abandoned. I'm cautiously interested in inflicting it on the world again, but somewhat stymied by a combination of insufficient documentation (gee what a shock) and the weird state of my coding style at the time, partly because of the state of the Python language and libraries. That can only shoulder so much of the blame, though. I'm looking at this code and going "what's the point of showing off like this, if nobody's going to have the patience to understand it?"

Anyway, I think the sensible course of action is to try switching course to writing tomorrow. We'll see if I feel like that. For now, I should get to bed.

Good night.

Diary 2024-06-29

2024-06-29T04:00:00-04:00

Okay, I'm still feeling kind of iffy and out of it, so I haven't made any more Dafny progress today. I messed around with some other code, but I'm probably not going to take it too much further right now.

I'm not up for thinking too hard now, so I'm just going to take things easy for a bit.

Good night.

Diary 2024-06-28

2024-06-28T04:00:00-04:00

I was thinking about Dafny stuff, but I didn't really make progress. I felt kind of iffy today, so I ended up just taking it easy, and suddenly realizing that it was much later than I thought.

Nothing good can come of trying to make this entry longer.

Good night.

Coding 2024-06-27

2024-06-27T04:00:00-04:00

I was focused on other things today, but I'm going to try to devote at least a few minutes to understanding what is up with Dafny.

According to the descriptions of the different batches when I isolated the assertions, the problem with the lemma that isn't brittle because of RNG screw is down to the postcondition.

A little more detail. The commutative property of summation basically says that we can re-index a summation, and get the same result, as long as we obey some "obvious" rules for the re-indexing. In the corresponding lemma, I'm using an ensures clause to assert this equality. Now, intuitively, I think the (relatively) high variance here corresponds to "requiring a lot of work from the verifier", which means I may be missing a step that Dafny is papering over. The current set of assertions is:

If we remove k from the initial set of values to sum over, the equality still holds.
Every item in the reduced set of permuted values "came from" a unique value in the reduced set of values.
The sum of the set of permuted values is equal to the sum of the reduced set of permuted values, plus the value indexed by the permutation of k.

There is a missing step after this that Dafny can, "with effort", fill in for me. My hope is to figure out how to guide it so it can just go.

I'm not getting there tonight, so I'll get to bed and think about it for a while.

Good night.

Coding 2024-06-26

2024-06-26T04:00:00-04:00

I've gotten everything currently written to verify, loosened some preconditions, and worked on reducing brittleness. At this point, there's just one brittle verification left, following some very subtle changes.

...

Following my efforts to decrease brittleness, there are now two brittle verifications left.

The RNG screw is so, so real.

In any case, there should be just a bit more effort required to get this locked back down, but I can't put that effort in right now. I've got to sleep.

Good night.

Weekly Roundup 2024-06-25

2024-06-25T04:00:00-04:00

Wednesday: I got back into messing around with Dafny.
Thursday: I gave some thought to code organization.
Friday: I decided to get a fresh start with the code, and try to get the tooling as nice as can be managed right now.
Saturday: I tried to make my Dafny code more reliable. In other words, to mitigate the effects of bad RNG.
Sunday: Those efforts were impeded somewhat by the stark simplicity of some of my proofs. (Dafny verified lemmas with empty bodies.)
Monday: I ran into trouble proving the commutative property of sigma-summation, because Dafny wanted me to "do actual work" and "actually think about the problem".

Next week, now that I've made some progress (that wasn't written up), I'm going to catch up on documentation, research writing tests, and then circle back around to reducing brittleness.

Coding 2024-06-24

2024-06-24T04:00:00-04:00

Okay, I made some more progress with Dafny, and now I need another break. I reorganized things a little to make imports nicer, then made some improvements that might make those changes unnecessary for that purpose, but they still help in other ways, so I'm keeping them.

I got everything together for the commutative property, and now I've got the problem that, where the other properties just instantly verified, the commutative property just spins indefinitely, without any indication of "what is wrong".

It's late, and I need some time to figure out what's going on. I'm going to wind down now.

Good night.

Coding 2024-06-23

2024-06-23T04:00:00-04:00

All right, things are moving along with working with sums. I've "proven" the distributive law and the associative law.

"proven"?

Dafny was all like "yeah, that's obvious, you don't need to write anything in the lemma body".

Anyway, the stuff I did to set that up does not quite transfer to the commutative law, so I need to do some more planning.

...

Also, I need to figure out how correctness-checking a lemma with an empty body can be brittle. Like, what? What's left to optimize?

...

Okay, I actually was able to squeeze a bit of brittleness out by making the preconditions more explicit. But still not all the way.

Anyway, that's it. I don't have anything else in mind, and I need to think and plan some more. I'd also like to get off my laptop.

Good night.

Coding 2024-06-22

2024-06-22T04:00:00-04:00

Dafny is a land of contrasts.

I'm getting code ported over, and being careful about where to put it, and working to make it less brittle, but unfortunately... Sometimes changes that I think will make the code less brittle, or at least easier to understand, make it more brittle, which is unfortunate. And it turns out that some of these extra diagnostics increase the resource usage and the variance, so I can't really use them unless I figure out how to tune the code even harder. And it's not even always obvious which code will be brittle.

How it's all shaken out is, I've gotten my finite sum code to parity with the original version, but I'm feeling some aesthetic quibbles, so I'm going to try to polish this some more tomorrow. Then I can work on some of the more interesting ideas I wrote down over the past week.

Anyway, I'd like to get some sleep now.

Good night.

Coding 2024-06-21

2024-06-21T04:00:00-04:00

Okay, I've laid some groundwork, so now let's lay some more groundwork.

I've decided that I want to try to get a fresh start on the Dafny code that I want to work with, so I'll create a new folder to work with it in, set up source control, and put together command-line hooks and task runner configuration to take advantage of capabilities in Dafny that I haven't used before.

The first thing I need to keep in mind is that I have to make sure verification is bounded. In the past, I used --resource-limit 200. I don't think that's any kind of generally applicable setting; probably if I go that route, I'll have to gradually ratchet it up, but as long as I just configure it in one place, I think that's fine?

Let's take a look at the kind of stuff I want to have in here, long-term:

commit hooks to run verify, then format check
warnings for common issues
workflows to diagnose common issues
documentation building
probably other stuff

I've just written a small module, and I'll be able to build up the various diagnostics now, and then start expanding on it.

For now, I want to get off my laptop, so I'll just note that I'm planning to look into just, to see if I prefer it to invoke, just out of curiosity. Right now, though, let's wind down...

Good night.

Coding 2024-06-20

2024-06-20T04:00:00-04:00

I think Dafny has gotten some new capabilities since I last messed with it, which is nice. I'm still getting to grips with stuff like "how should the modules be laid out?", and I kind of feel like maybe I should try getting a fresh start on this code.

Quick notes on the organization, because I was spacing out thinking about something else, and it's late:

There are functions that are good for reasoning about sums.
There are functions that are good for reasoning about those functions.
There are methods that relate those functions to actual code.
There are lemmas that allow the implementation of the method, and lemmas required to relate the method to the function.

It's probably safe to not try too hard to surface the lemmas, but I'm not sure how best to lay out everything else. I'll have to think about this some more. (I was thinking about Dafny stuff earlier, but it was mostly about a more sensible approach to the higher-order function ideas I had earlier.)

Anyway, that's all I've got for now.

Good night.

Coding 2024-06-19

2024-06-19T04:00:00-04:00

I've picked Dafny back up, updated it, deleted some asserts that are no longer required, (continued to despair that some asserts are required specifically to avoid z3 going into an infinite loop), and reviewed some of the code I wrote however many months ago. Because I didn't comment anything at the time, I had to look over the code and remember what the heck I was trying to do with all of it. Before I write any more code, I want to make sure to document this stuff thoroughly, so if I put this down and pick it back up, I can get up to speed in minutes rather than hours. I also think it might make sense to have separate modules defining deterministic and non-deterministic sums, but I'm not sure about that. Like, in my head, non-deterministic sums have a simple definition and support most theorems, and then deterministic sums "automatically" convert as many of those theorems as possible into theorems that apply to "method" contexts.

I'll just end up confused if I move hastily here, so I'm going to keep thinking about this, and focus on documentation, as I said. For now, I need to get ready for bed.

Good night.

Weekly Roundup 2024-06-18

2024-06-18T04:00:00-04:00

Wednesday: I made more progress clarifying how code in MOTR relates to other parts of itself, and, indeed, how it doesn't.
Thursday: I got all the way to the final logical step in all of the work I was doing there, and then paused in case I was making a bad decision somehow.
Friday: Nope, perfect idea. And for a followup... certainly a concept.
Saturday: Switched gears to inlining a hierarchical... singledispatch... thing... The code is shorter and easier to understand, that's the important thing.
Sunday: I finished with all of these distractions, and then had to face up to increasing coverage somehow, which, man...
Monday: I guess I've got to actually implement new functionality, what a pain.

Next week, I will carefully plan the new wrappers in MOTR, and not rush in. I'm also thinking of going back to messing around with Dafny.

Coding 2024-06-17

2024-06-17T04:00:00-04:00

All right. I've made some probably-valid changes that increased code coverage.

I've also written up what I think needs to be done to address the remaining coverage misses.

I think the correct way forward is to start writing more wrappers, and make notes of what needs to be changed to make them easier to use. That means starting to write coverage wrappers. The order of those should be something like:

erase
run
combine
everything else

I should also check whether any new subcommands got added. In any case, this should remind me how I thought all of this was supposed to work, because I'm pretty sure some of the functionality I've written is meant for the kind of workflow I've put together here.

Anyway, I'm not interested in being on the laptop much longer.

Good night.

Coding 2024-06-16

2024-06-16T04:00:00-04:00

Okay. I've inlined and enhanced the function I was working with yesterday, and now the code is fully ready to be properly covered.

But, like, I kind of don't want to? There are a few dozen lines of missing coverage currently, and I just look at them all and think "Ugh, how confusing, what a pain."

At this point, either I can, like, try to make myself eat my vegetables and grind through the lines, ooor I can go over some of the notes I've taken on improving code from the standpoint of writing tests that involve emulating a high-level motrfile. The best hope I have for justifying that is, "Well, if I feel better about writing more tests, then I'll be ready to write tests for new components, and I can use those components to increase the coverage because they'll exercise more of the API surface, probably".

It's too late right now for me to try to tell whether that idea is, like, valid, so I'm going to get ready for bed now.

Good night.

Coding 2024-06-15

2024-06-15T04:00:00-04:00

Okay, let's see... I did a bunch of code reorganization today, and now things are generally in, I think, nicer shape. I cut down on some duplication, and I've gotten most of the way through a rework of one of the problematic modules. What I need to do now there is to put in a few more isinstance guards, which should make sure I'm covering every case that I'm worried about.

I don't quite have the wherewithal to add them now, so I'll try to remember for later.

Anyway, I want to shut the laptop down soon and start moving towards bed.

Good night.

Coding 2024-06-14

2024-06-14T04:00:00-04:00

Okay, I feel better. I wasn't extremely focused on coding today, but I did get some work done tightening up the protocols like I was talking about. I've moved around some code, and deleted the rest, and the result all seems to make sense.

I still need to actually work on improving the coverage. Somewhat unfortunately, I'm considering doing something bizarre to increase the coverage. Like, I want to use queues or something to avoid having to have two different versions of map depending on whether Facts are involved, but this could be a bad idea. I need some time to think about this, as well. For now, I once again really need to get to bed.

Good night.

Coding 2024-06-13

2024-06-13T04:00:00-04:00

The Combinable protocol is now truly filtered down to the smallest possible scope, and now I want to figure out whether it's worth having as a separate protocol, or simply folding it into the only thing that "really" uses it, which is... another protocol. It looks like a slam dunk, but I'm not completely sure, so I'm just going to try to get a good night's sleep. (My antihistamines have been giving me bad night's sleeps, so I'm skipping a day and trying taking them at a different time.)

Anyway, I also have an idea for stuff to do differently with the whole Crafting Interpreters in Rust idea, which is to ignore the nitty-gritty of the code to start with, and try to fill things in with idiomatic Rust. I need some time to work through that idea as well, though. So, bed.

Good night.

Coding 2024-06-12

2024-06-12T04:00:00-04:00

Okay, I believe I've winnowed down the Combinable protocol usage/implementation to where it's actually relevant. Now, I need to consider whether to rename these, and then I can decide whether to do the changes I was thinking about however long ago.

In any case, it would be good for me to get to bed, and I'll get to work on this, and other things, later.

Good night.

Weekly Roundup 2024-06-11

2024-06-11T04:00:00-04:00

Wednesday: The whole "the tutorial said to do X, but I did Y" thing isn't cute any more, if it ever was.
Thursday: Nevertheless, I did make some progress.
Friday: I sort of made my code closer to the tutorial.
Saturday: I accidentally made friends with the borrow checker.
Sunday: I noticed that it wasn't cute.
Monday: I went back to MOTR, but was hindered by hiccups.

Next week, I'm going to keep on poking at MOTR, but also do various bits of writing that may or may not appear somewhere.

Coding 2024-06-10

2024-06-10T04:00:00-04:00

Bad news, everyone. I accidentally gave myself really bad hiccups, and it's hard to focus on code or writing when I am constantly jerking my head backwards like I'm possessed by a demon.

So, to very briefly summarize what I was able to accomplish with MOTR today:

I took stock of my implementations of a Protocol that has some issues, and determined that most of the implementations would probably work fine without worrying about the protocol, so I started renaming methods. This represents some delicate work, because all of a sudden I have to think about what they should be called, if they no longer implement the ThingDoer protocol via the do_the_thing() method. I got some of those done, but there are only so many easy ones, and some of the remaining ones are tricky to think about.

Anyway, I'm going to go try lying down. Maybe that'll cure me.

Good night.

Coding 2024-06-09

2024-06-09T04:00:00-04:00

I can't speak for anyone else, but right now, I'm thinking that reading Crafting Interpreters, and then rewriting all of the code on the fly, is not a great way for me to learn Rust. Still not really up for actually touching Java outside of my senior thesis and I Minecraft mod I tried to maintain around the same time, though.

So, I guess I'll ponder that, and try to get back to MOTR later, since at least with that, I'm not putting a stick through the bicycle wheel as much.

Regardless, step one of whatever I decide to do, is to get some sleep.

Good night.

Coding 2024-06-08

2024-06-08T04:00:00-04:00

So, how are things going with your programming hobby proects?

... I'm making a lot of progress in Isles of Sea and Sky?

... Of course you are.

Anyway, I made a little progress on the Crafting Interpreters stuff. Rust is letting me get away with a lot of stuff that I kind of didn't want to do, so I think I'm going to need to learn to trust the compiler more when it comes to data ownership.

Just how many blog posts and explainers did you have to read that, when you finally started putting together a proper project in Rust, your reaction to the borrow checker was "Wow, it lets me do so much stuff!"

Probably too many. Anyway, I meant to get to bed, like, half an hour ago basically. I'll look at this, or at something, tomorrow.

Good night.

Coding 2024-06-07

2024-06-07T04:00:00-04:00

I'm right now redoing things a little to make the scanner more like the Java version, because I realized something: all of this is encapsulated away from the main interface, so, as long as it doesn't leak into the front end, I can pretend it's not "messy" because it has a clean interface.

What I'm thinking about now is whether the scanner should take a mutable reference to the interpreter, for the purpose of error reporting. I'm like, I'd like to see how that shakes out, but I'm currently dealing with... some consequence of the weather, I think.

Speaking of which, my thoughts aren't currently moving super well right now, so I'm not going to try to write any more right now.

Good night.

Coding 2024-06-06

2024-06-06T04:00:00-04:00

Still playing with the new toy, so I didn't have much time to touch code today. (I think I might need to just take a break from trying to blog for a bit? I'll think about it.)

Here's where I'm at thinking about Crafting Interpreters:

The scanner needs to take slices of the input string to do various things, but the builtin char_indices stuff returns the start index of each code point, which means I need some way to go from "an index on a code point boundary" to "the next such index". This can work with is_char_boundary, which should check no more than four indices ahead I think.

The challenge is to take these ideas and use them in a way that makes sense.

Anyway, it's late, I should get ready for bed.

Good night.

Coding 2024-06-05

2024-06-05T04:00:00-04:00

I was thinking about Impliciula more recently, so I've been revisiting Crafting Interpreters again. As per usual, the biggest obstacle I'm facing is my stubborn refusal to follow the tutorial in some crucial way.

I'm right now trying to handle the scanner in Rust, and it's fiddly enough right now that I'm considering prototyping it in Python, and then porting the logic over.

Anyway, it's late, hot, and I have other stuff I need to work on, so I'm going to cut this entry off now.

Good night.

Weekly Roundup 2024-06-04

2024-06-04T04:00:00-04:00

Wednesday: I reviewed some of the low-level parts of MOTR.
Thursday: I started focusing on improving coverage again.
Friday: I made some progress, and then had to face weird questions.
Saturday: I made more progress, but was hindered by how confusing some of the code I wrote is.
Sunday: I closed some long-standing coverage holes, thereby uncovering some new bugs.
Monday: I reviewed some of the high-level parts of MOTR.

Next week, I'll try to keep on making progress, but I've got a new toy and started playing a new game, and they're pretty absorbing.

Coding 2024-06-03

2024-06-03T04:00:00-04:00

I did a little work improving coverage today, but I ended up spending most of my effort describing a test that shows off some of the issues with the current top-level interface, and then messing around with other things.

I'm not up for describing those other things right now, but maybe in a few weeks...

Anyway, I'm tired and I should get to bed.

Good night.

Coding 2024-06-02

2024-06-02T04:00:00-04:00

Well, I added a bit more test coverage to MOTR, and the additional coverage, say it with me, uncovered a horrible bug that was latent in the dead code paths.

I just got a little bit more coverage on top of that, but at this point, if there are any easy wins left, then finding them is going to be hard.

Anyway, it's late and I want to get to bed, and I never actually did the thing I wanted to do this morning. Tomorrow or the day after, I suppose.

Good night.

Coding 2024-06-01

2024-06-01T04:00:00-04:00

I picked away at the coverage misses in MOTR for a while, and it's definitely doing better now.

One problem I ran into is that the circumstances in which some of these lines of code should be hit are, at first glance, frustratingly vague. At some point, I'll have gotten coverage on all of the lines that basically make sense, and then I'll be stuck documenting all of this for myself until I can understand what's going on.

Anyway, it's late, and in the morning I've got some stuff that I absolutely need to be awake for. So, I should get to bed.

Good night.

Coding 2024-05-31

2024-05-31T04:00:00-04:00

I've gotten full coverage on a few files, and now the question of how to deal with the next file is interesting. I've got some slightly questionable functools.singledispatch calls that I'm not sure what to do about. I'll have to think about those.

For now, I'm tired and I want to sleep.

Good night.

Coding 2024-05-30

2024-05-30T04:00:00-04:00

All right, I took care of a few coverage misses. In particular, I'm now using saltate in the places that I think it made sense, and I've added a missing conditional to the plugin to deal with the realities of usage. For now, I'm improving coverage starting with the lowest coverage percentage per file. By that methodology, there are fifteen files left to update tests for, although at some point I'll bump back into "the most natural way to increase some of this coverage is to add more CLI wrappers", which is going to get me back on "it feels like there should be a better way to create CLI wrappers".

I'll figure that out when I figure it out; for now, I'm tired.

Good night.

Coding 2024-05-29

2024-05-29T04:00:00-04:00

I'm mostly trying to explain MOTR to someone else, so today I just worked around a bug in Mypy, or possibly its bundled typeshed install. The one place I use functools.wraps in this code base, and it breaks Mypy.

Anyway, I got that in, and now I'm pondering some parts of MOTR that I haven't needed to in some time, because they're so solid and reliable.

This is a badly written plug for the trio library.

It's pretty nice, and is probably the main reason I barely need to think about the core code running MOTR.

Anyway, I don't want to be up any longer.

Good night.

Weekly Roundup 2024-05-28

2024-05-28T04:00:00-04:00

Wednesday: I noticed some interesting developments in Koka, and promptly mostly ignored them.
Thursday: I did a small refactor to MOTR.
Friday: I did another small refactor.
Saturday: I wrote about stuff that Impliciula could do.
Sunday: I took a look at what's left to clean up the names in MOTR.
Monday: I took a closer look; I'm still not quite sure how to address these issues.

Next week, we'll see what I'm up for. Earlier today, I did the big unpleasant merge that's been a long time coming, and I may have created some duplicate code; it's legitimately unclear at this point.

Coding 2024-05-27

2024-05-27T04:00:00-04:00

Taking another look at the cli_types package...

artifact: High-level code for ... I'll get back to you on that.
combinable: Protocol for a generic magma.
entry: Small module with a set of marker types.
environment: Small module for... something looks weird here... Okay, so there's a type-level marker, but it's attached to a small dataclass that handles resolution within a virtual environment? And this marker is, at least in part, propagated via a protocol class? I suppose a good condidate for stuff to bundle with this would be "whatever implements the protocol?" Except that it's not implemented anywhere in cli_types. It's implemented once, which, shh... shh...
executable: Tiny module for specifying a program entry point accessible in an abstract way.
installer: Does stuff with environment.
invocation_environment: Ties together a bunch of this stuff as well as lower-level stuff, to represent an abstracted command argument, and the installers needed to make it specific.
not_output: A helper function for preventing various data structures from containing an io.Output value. I wonder if it makes sense to decouple iteration from instance checking?
parametric_command: Due for renaming to "command_matrix" or something. I think this is the thing that the cli modules use to represent the basic form of a command, which isn't really a clear description.
path_with: Class to associate some data to a path. I should consider whether I want the path to default to anything in particular.
program: Marker class.
runner: Another part of the whole "tower of abstraction" thing.

It's late, so I'm not going to touch anything tonight, but it looks like creating a package to bundle together marker types might not go amiss. Then again, it might. Regardless, the big division that's manifesting for me is the "tower of abstraction" vs "all the other stuff". Perhaps it would be illuminating to lay out the "tower", and figure out which levels "everything else" has to go "between"? I'll try that out later, I don't know exactly when.

All that said, I'm very worried about the scope of the merges I'm setting myself up for, so it would also be a good idea for me to review my coverage misses and see what's in the way of addressing them...

Anyway, time to stop messing around and get ready for bed.

Good night.

Coding 2024-05-26

2024-05-26T04:00:00-04:00

Well, I don't have much to write about—

On account of all of the video games.

And travel.

But I did just now break out the matrix modules on their own, which means that that's three fewer modules in the badly named cli_types sub-package. Just twelve more to go...

(Also, I should probably clean up all of the .orig and .rej files that are cluttering things up from all of the merges and whatnot, but I'm putting that off until after the Big Merge that all of this is working towards.)

Anyway, it's getting late and I need to get to bed.

Good night.

Coding 2024-05-25

2024-05-25T04:00:00-04:00

Tonight, I ended up doing a bunch of random things, then focusing on Impliciula. Specifically, I made a post, on Cohost, so it's easier to comment on it, taking a simple Rust function with a one-line body, and extracting a large number of questions from it about how Impliciula should achieve something similar.

It's pretty late at night, so I don't want to think about this too much harder. Let's just wrap things up.

Good night.

Coding 2024-05-24

2024-05-24T04:00:00-04:00

All right. label is at the top level. In the utopian future I'm working towards, when dependent_mapping gets spun out into its own package, label is coming along for the ride.

I wanted to have more to say, but I evidently don't, so I'm going to get ready for bed.

Good night.

Coding 2024-05-23

2024-05-23T04:00:00-04:00

I've been thinking about it long enough, so I finally went and removed a bunch of dependency links in MOTR. I think this resulted in some names not quite making sense, so I'll just let the new status quo percolate a little.

However, now that I've made the change, I'm ready to move things around. I believe I'll want to make label a free-standing top-level (or at least higher-up) module, and consolidate the rest of the matrix stuff into a top-level package. Anything to make this all a bit more coherent...

Anyway, I've been feeling tired all week because of reasons that are understandable to me, and I need to suffer through a few more days I guess, but I know when it should be over.

Maybe I'll get much more done then, maybe I won't. We'll see. For now, I should get to bed as soon as I can.

Good night.

Coding 2024-05-22

2024-05-22T04:00:00-04:00

So, Koka has had implicits since sometime in January.

Oh wow, neat.

So, there's no reason to keep messing around with Impliciula, right?

Well...

Why do I bother?

See, here's the deal. There are various ways to handle the concept of implicits. I haven't delved super-deep into how Koka's implementation works, but it seems maybe like it's closer to Scala than I'd like; my goal is to make something much more like this one particular fork of OCaml.

Of course.

Also, while the research programs feeding into Koka's development don't have any way to crowbar in AI BS that is extremely obvious to me, just thinking about that somehow happening gets me thinking about stuff like "a healthy ecosystem with plenty of competition". (To be clear, I'm not talking about "having libraries that do a lot of linear algebra and vectorization", I'm talking about "Buy a fancy new computer that records everything on your screen and keeps it around for three months by default." HIPAA violations as a service.)

Anyway, I've got some ideas for Impliciula that I've been posting on Cohost. I think what I'd like to do is to take code samples from language tutorials (starting with The Rust Book), and figure out either what they would look like in Impliciula, or what questions I would need to answer first.

For the moment, I need rest.

Good night.

Weekly Roundup 2024-05-21

2024-05-21T04:00:00-04:00

Wednesday: I got tup to run a basic lexurgy sc command, and started thinking about how to do fancier stuff in that area.
Thursday: I spent a bunch more time thinking about how to glue together a bunch of lexurgy sc invocations with tup.
Friday: I stepped away from code and project structure to focus on the naming language itself.
Saturday: The scope for the naming language ballooned.
Sunday: I started to run into difficulties.
Monday: If a sound change generates a sequence of phonemes that has to be transcribed something like "ththth", it's probably a bad sound change in context.

Next week, I'm going to step away from conlanging, and get back to broader world-building. Maybe I'll finally mess with MOTR again, as well.

Conlanging 2024-05-20

2024-05-20T04:00:00-04:00

Well, after a bunch of effort, I've got some attempts at names in a naming language, and boy, do I not like it. It looks extremely likely that I have fundamentally misapprehended some aspect of Proto-Indo-European.

So, now I'm trying to just come up with a different set of sound changes, and I'm not sure how to get from the source language's syllable structure to my goal. I'm going to either put this down for a bit, or at least get more research in.

We'll see how that goes, but I want to wind things down just now.

Good night.

Conlanging 2024-05-19

2024-05-19T04:00:00-04:00

Now that I've got some idea of how Proto-Indo-European worked, I'm trying to run things forwards with the minimal selection of roots I need to generate the names that were the whole point of this. In doing so, I have learned two things:

I can read some ambiguity into Wikipedia's description of the idealized regular ablaut.
I don't know what to do when I put a syllable in zero-grade, and then the multiple sonorants mean it doesn't have a defined nucleus in my eyes, and then I'm supposed to accent it?

That said, I've run a few roots partway forwards, and the results so far look interesting. We'll have to see whether things still look good once I've figured this out.

Anyway, I want to cool off and get to bed now.

Good night.

Conlanging 2024-05-18

2024-05-18T04:00:00-04:00

All right, before I get into the details of what I'm planning, I'm just going to ask that everyone ignore the degree of scope creep that I've inflicted on this project. All right? All right.

So, here's what I've worked out so far during my attempts to create basically an alternative version of an Indo-European language with a different vocabulary of roots:

Need to work out ablaut before applying other sound changes.
Need to determine accent before applying ablaut.
Because my goal is a centum language, I can't totally nail down the roots unless I also do derivations for a satem language.
(This perhaps could be avoided by attempting to make an alternative version of Luwian or certain other Anatolian languages, but, I'm not doing that. (Right now.))
My original plan was to use Farsi for that purpose, but I'm having trouble tracking down sets of sound changes in general that go into as much detail as Wikipedia goes into on Proto-Germanic. I may need to just look harder.
On the other hand, I could just learn the tools with Proto-Germanic, and then try to do things based on Vibes for the rest of it.

At this point, it makes sense to try to put together the basic rules, but it also makes sense to try to get more of a handle on derivational morphology, so I have more idea of what can cause the accent to move around.

I'll see what I'm up for in this area after I get some more sleep.

Good night.

Conlanging 2024-05-17

2024-05-17T04:00:00-04:00

Okay, I've done some work on the phonological evolution side, and I've realized a few things. Most importantly, the Index Diachronica isn't really useful to me, not yet. This is because, and I swear I'll come up with a good reason I didn't notice this before, it present the sound changes in a given section in lexical, rather than chronological order.

Figuring this all out myself does mean that I have more of a handle on how the sound changes actually fit together, where to look going forward, and also just the proper way to make up Proto-Indo-European roots, and to apply ablaut.

At this point, my plan is to make sound change files for regular ablaut application, and early and late Proto-Germanic. At that point, I can check that they actually work by comparing them with actual historical derivations. And then just keep going until I've run it forward enough.

At that point, I can start experimenting with fictional roots, and get a sense for stuff like "if I need a laryngeal to fill out a consonant slot, which one do I prefer?" and "do I get nice results if I give this root a laryngeal, as a treat?" and "which plosive would I like to use?" I can try to describe what conclusions I come to there, but it will probably end up being very personal/tacit knowledge. Still, I'll see what I can do.

Anyway, it's getting late, and I want to wind down. Hopefully, I'll make some progress on this tomorrow.

Good night.

Coding 2024-05-16

2024-05-16T04:00:00-04:00

All right, here we go. I figured out a basic nice interface for defining Lexurgy invocations in a Tupfile.lua, but I'm not going to be using it.

Of course you aren't.

Nevertheless, I'll start off explaining it, because it makes a good starting point for the fancier stuff I have in mind.

Think of the development of a language from a protolanguage. Since I'm aiming for Old Norse, let's see what the conception of the sound changes looks like:

Apply ablaut to increase the number of vowels.
Proto-Indo-European to Common Germanic
Common Germanic to Proto-Norse
Proto-Norse to Old Norse

Now, I could stuff all of those into a single file, but I like the idea of keeping the sound changes divided, so I could, in principle, branch off from somewhere and create other members of a language family. So, rather than the single change that I specified to carry out the Basican example from Lexurgy's documentation, I need a chain of related changes for this. And those changes have some things in common, and some things different. And some of the differences are regular: the output suffix of one step must be the input suffix of the next step. Now, there are plenty of ways to express "a bunch of similar arguments" in Lua. Table arguments, varargs, probably other things than those and what I have in mind. But here is what I have in mind:

lexurgy = lexurgy or {}

local wli_template = "%s_%s.wli"
local wlm_template = "%s_%s.wlm"

do
  local function word_list(name, in_suffix, in_build_folder)
    local in_prefix
    local out_prefix = "build/" .. name
    if in_build_folder then
      in_prefix = out_prefix
    else
      in_prefix = "src/" .. name
    end
    return function(out_suffix, sound_changes)
      tup.definerule(
        {wli_template:format(in_prefix, in_suffix), sound_changes},
        "lexurgy sc --in-suffix " .. in_suffix .. " --out-suffix " .. out_suffix .. " --out-dir ../build " .. sound_changes .." " .. in_prefix .. ".wli",
        {
          wli_template:format(out_prefix, out_suffix),
          wlm_template:format(out_prefix, out_suffix),
        }
      )
      return word_list(name, out_suffix, true)
    end
  end

  lexurgy.word_list = word_list
end

Wow, I... genuinely do not know whether I'd rather build that command line with string concatenation or a format string. Python has spoiled me.

Anyway, code very much like this should work, if someone is reading this and thinking "By gosh, yes, I would like to coordinate my sound change application by simply running tup, and I'll have no truck with the other bells and whistles this post is about to add!"

However. My end goal is to generate a lexicon from this stuff, and that means not just having the evolved form of the word, but also information about how it's used. A specific line of a .wli file corresponds to a precise lineage of word, spread out across all files where the filename starts off the same. This means that, as long as I figure out a format for this metadata that all fits on one line, I don't need to further process such metadata as the sound changes are applied.

So, this is going to be a simple change, if it's any change at all, right?

Thank you for teeing that up. While normal languages get most of their vocabulary through evolution of ancestral word forms, loan words are a thing, and it would be nice to bring in and evolve them. At this point, I can see two approaches to take with that:

Concatenate the word lists and metadata together, creating a new single lineage, and process them as a single block at the end
Have an array of word lists separated by origin, and combine them at the end

The former relies on the fact that the inputs to the word_list function above are sufficient to uniquely determine the location of the word list file being processed (otherwise, it couldn't process it). That means that it could "make sense" to replace the function returned from word_list with an object representing the file, and a method representing the original function, and other methods for other operations.

That was my first idea, but the more I think about it, the more I don't like it. Coordinating the concatenations feels fiddlier than I'd like right now, and requires exponentially many names for lineages. Therefore, I think I want to consider the outcome of handling lists of lineages. The in_suffix should be the same for all of them, so, hmm... Hmm...

I think I would like to finish this entry now, and ponder further after publishing.

Good night.

Coding 2024-05-15

2024-05-15T04:00:00-04:00

The good news is, I've managed to prototype an invocation of Lexurgy SC via tup. The not as good news is, it looks like Lexurgy SC's somewhat idiosyncratic file naming paradigm is going to require the use of Lua to actually generate the correct names, if I want to make this stuff parametric and reusable, which I do.

Let's take a look at what I had before I switched to Lua:

: src/words_old.wli |> lexurgy sc --in-suffix old --out-suffix new --out-dir ../build src/changes.lsc src/words.wli |> build/words_new.wli build/words_new.wlm

There are several things that can reasonably be varied:

The out suffix
The in suffix
The path to the changes file
The prefix to the word lists

Let's make a few assumptions:

The user should specify the path to .lsc files exactly, since there's no funny business with the file names.
.wli files reside in src and build, nowhere else.

From this, we have the following desired inputs, probably not in this order:

Out suffix
In suffix
Input directory
Input filename
Sound changes

I'm going to need to think some about the exact format that I want some of those to be in. For now, I'm going to bed.

Good night.

Weekly Roundup 2024-05-14

2024-05-14T04:00:00-04:00

Wednesday: I dusted off the Conlanging category. And promptly ended up making a bunch of work for myself by deciding to make a naming language with ablaut and umlaut and probably various other kinds of -laut.
Thursday: I thought some about how I want to handle the development of the naming language in a hopefully reusable way.
Friday: I decided to drop marimo in favor of Pandoc and tup for my grammar needs, which is a really messed up statement if you lack a whole bunch of context.
Saturday: I gave some thought to learning Lexurgy.
Sunday: I gave more thought to specifically using tup and Lexurgy together in a way that at least sort of makes sense.
Monday: I put down in writing some more of the thoughts that I'm having there.

Next week, I'm either going to keep on working on developing Tupfile stuff to handle the Basican example from Lexurgy, or I'm going to pivot to reading up on Fabula Ultima rules again, and try to figure out which oracles I want to bring in.

Coding 2024-05-13

2024-05-13T04:00:00-04:00

Okay, let's see. For the lexurgy commands, I want to be able to input a word list either from the src directory or the build directory, and it needs to output to the build directory. For consistency, I want both lists to have a suffix in the filename, so to get tup working, it's going to need to do a little string processing on for both input and output, in some fashion. Not sure offhand how well tup takes to that, but oh well.

The src directory also needs to maintain a mapping from ancestral words/phrases to modern definitions, so there can be a parallel list that puts all of those in the same order.

A big part of what I have in mind is going to literally just be "here are two files; cat them together and write the result to the build directory".

That said, I think what I need to do to try this out is to write ports of the Lexurgy examples. I just cloned a repository for one of them, but my feeling about what I want to do right now is "We were back on the road today, so I'm going to go lie down early".

Good night.

Coding 2024-05-12

2024-05-12T04:00:00-04:00

I've thought enough about how I want to lay out the repositories for conlangs that I was able to handle part of the updates. I've got two tracked directories at the top level: src and utils, as well as a bunch of files specific to tup. The idea is to somehow convert from one of Lexurgy's .wli files to a basic .rst file containing a table and an .. only:: html, .. raw:: html directive or similar, to embed some javascript to make it interactive in the HTML version.

Getting all of that to work right/at all is going to be more effort than I've put in so far, and I can't handle that right now. But soon, hopefully. Anyway, it's late again, so I need to shut things down.

Good night.

Conlanging 2024-05-11

2024-05-11T04:00:00-04:00

Okay, let's see what I've learned so far. There are sort of two sides to learning Lexurgy.

There's actually writing the rules, which takes skill working with groups, feature matrices, etc.

And there's figuring out how I want to wire the word lists together. That gets into stuff like how I want to specify things in the configuration files for build systems (tup, Ninja, ... Make...), and what I want to track in version control (Pijul, Mercurial, ... not saying the name of the popular one...).

I can get a better idea of what to do for the former by studying stuff like the Index Diachronica, though I wouldn't mind finding an alternative, because some of the very early sound changes for Proto-Indo-European look to me like they're written against a reconstructed phoneme inventory different that the one I want to go for.

Anyway, that's all going to take some more thought, and I want to wind down for tonight.

Good night.

Coding 2024-05-10

2024-05-10T04:00:00-04:00

Okay, I've done some investigation, and I have a fair idea of what won't work for what I'm trying to do here. The key thing I figured out is that marimo doesn't really work for putting together a conlang grammar, not the way I'd like.

See, one of the things that I was interested in was just shoving together some widgets to handle fancy table stuff like filtering and sorting by column, and it turns out that "interactive widgets that depend on running Python code for their functionality" doesn't really have a great way forward in marimo. Too many tradeoffs that I don't really like. That said, I don't see any problem with continuing to use it for the optional interactive IPA thing.

Here's what I'm thinking I will do:

Put together a basic scenario to make sure I understand how to use Lexurgy
Get some hands-on experience with tup, because it seems to have some features that Ninja doesn't, and I'm curious
Investigate pandoc's support for including files using reStructuredText
See about generating reStructuredText tables
Try to get pandoc to emit fancy table manipulation javascript when building HTML

My current idea about how this all ends up is to have some code that ultimately generates the lexicon (this can get pretty fancy in various ways), and possibly other sections. This feeds into a single reStructuredText file via includes. Somewhere along the line, some javascript libraries get linked into the page. The reStructuredText file gets converted to, for example, HTML and PDF, using pandoc. All of this gets glued together by tup if I can get it working, and Ninja if I can't get tup working. (I'm hopeful that tup works well, but until I've got more experience with it, I've got to allow for the possibility that there's some dealbreaker. Like how right now, marimo's HTML export puts all of the stuff specific to a given notebook into a single line of messed-up-looking text. Like, it renders mostly fine-ish, but it lacks the features I'd like.)

Anyway, that's planning done, now I just want to relax for a bit before getting to bed.

Good night.

Conlanging 2024-05-09

2024-05-09T04:00:00-04:00

Okay, let's see if I can figure out what I "should" be working on here.

I want to design an Old-Norse-analogue
Which is going to be descended from a Proto-Indo-European-analogue
Which could plausibly have other descendants, if I go that direction
I'm handling the combination of "common elements and specific elements" by creating a single source repository, and cloning from it

With all of that in mind, it seems to me that I could get sensible behavior by creating a generic "descendant of this protolanguage" repository, and then cloning a repository for Cosmopolitan from that. That deals with some concerns I had. Next up...

The source lexicon will be generally stable, and each root in it will be transformed by a set of rules that is consistent within a single repository
It seems to me that I should like to verify the form of a root at specific checkpoints, and handle stuff like sometimes synthesizing roots together somehow; somehow

That isn't resolving out as obviously as the previous list. Let's let that sit, and consider the issues I'm dealing with currently:

It's really obnoxious to construct phoneme tables in HTML
I'm not sure what's a good way to handle sound changes

I've got some ideas partially sketched out for the first one, so let's focus on the second one (assuming that the whole intermediate descendant repo idea works out). There are command-line tools for applying changes, and Python code for applying changes. I know I've got some kind of attempt lying around...

Oof, that code sure is code.

At this point, I should see how shelling out to Lexurgy works. Just stop thinking about trying to find, update, or port code. See how it all works, and then only mess around with other stuff if there's some kind of specific problem I can't see how to address.

It's late, so I'll see how this comes together in a few days. For now, I really need to get ready for bed.

Good night.

Conlanging 2024-05-08

2024-05-08T04:00:00-04:00

I just remembered I have this category, and I have no recollection of what I was doing in it last time, whoops.

Anyway, as I was saying before, I'm really unhappy with the names I've come up with for the fantasy species in my fantasy setting, so I decided to devise a naming language so they can have names that sound related to each other, distinct from existing concepts, but aesthetically grounded in the general vein of mythology they're drawn from.

Okay...

That mythology is, more or less, Norse mythology, so clearly the naming language should sound like Old Norse.

Okay...

And some major aesthetic features of Germanic languages derive from ablaut, so clearly, the naming language needs a proto-language inspired by Proto-Indo-European.

I'm calling the police.

(By the way, I'm calling it Cosmopolitan because I figure it might as well be the lingua franca of the setting, and I wanted a name that was different from "Common". This can be rationalized as "people didn't like the language's word for itself, so they used a descriptive adjective".)

Anyway, on the one hand, this is kind of an extreme idea, but on the other hand, I don't want to compromise on this.

Anyway, I'm not sure what's the best way to get started on this... I can just put together some independent documents or whatever, but I want a chain of derivations that generally hold, and I'm not sure how to validate that between lexicons. I should see if PolyGlot can handle this, but unfortunately, I was taking things easy before, and I don't have time to look into it now. In fact, I shouldn't be still writing this entry...

Good night.

Weekly Roundup 2024-05-07

2024-05-07T04:00:00-04:00

Wednesday: Out of it.
Thursday: I came up with an Extremely Sound And Valid reasoning for developing a naming language for the Fabula Ultima stuff.
Friday: Out of it.
Saturday: Out of it again, but I did realize I need to slightly change the approach I was taking to the Fabula Ultima stuff.
Sunday: I did the things that are needed to course correct there.
Monday: I recapped why I think I need a naming language, and did some research for inspiration.

Next week, I'm going to try to shore up my homegrown conlanging code, because it is currently dire.

Diary 2024-05-06

2024-05-06T04:00:00-04:00

Well, things went late today, but I have done some research for the naming language that I want to use for the Fabula Ultima setting. (This kind of thing probably wouldn't work in a normal game, but I'm going to try to play solo, which... may end up involving a prohibitive amount of bookkeeping. We'll see.)

Basically, I'm trying to get inspiration from Old Norse, to match up with the Dungeons and Dragons Standard Fantasy Races that make up part of the setting. I didn't really like my attempts to draw stuff directly from Old Norse, so instead I've just been researching the aesthetics, and I think I've gotten to the point where I understand things well enough to move on to my satisfaction. (This could have been a more detailed understanding, if the explanations I'd found about phonotactics had been a bit longer.)

Basically, I should have enough information on hand to put together a basic phonology, phonotactics, and structure for basic parts of speech, so then I can start playing around and coming up with word roots.

Anyway, I should have been in bed ten minutes ago, oops.

Good night.

Diary 2024-05-05

2024-05-05T04:00:00-04:00

Well, that wasn't too interesting. Basically, I'm starting over from the actual beginning of the "Press Start" chapter in Fabula Ultima, which before I had somewhat skipped over. I figure it's a good idea to have down in writing stuff like "what I hope to accomplish". At this point, there are some bits I can try and go over again, but I think I'm ready to give the "World Creation" section another shot.

I may try to make a simple naming language or something first, but at least I covered the very first of the session zero things this time.

Anyway, it's late, so I'm going to wrap up for now.

Good night.

Diary 2024-05-04

2024-05-04T04:00:00-04:00

Ugh. I ended up mostly zoned out today. When I got some last-minute focus, I decided to spend it on stuff like getting my Bandcamp wishlist cleared out.

I just realized that I did some somewhat major things "wrong" with the worldbuilding project I've been messing with, so I'm going to try to put together a fresh start. Not going to worry about that right now, just try and start over tomorrow.

Good night.

Diary 2024-05-03

2024-05-03T04:00:00-04:00

Well, I was kind of out of it today, so again, not much to say. I'll try to get things a little bit more together tomorrow, but no promises. It's late now, so I'm going to wrap things up.

Good night.

Diary 2024-05-02

2024-05-02T04:00:00-04:00

Well, it was another light day, but I'm making some initial attempts to address one or two of the issues I've noticed with the Fabula Ultima worldbuilding stuff. See, I've got a bunch of "generic Dungeons and Dragons" tier sapient species, and some somewhat out-there ones I made up. The "generic" species have horrible puns embedded in their worldbuilding, and I don't get all of them, and don't want to get rid of the worldbuilding side of the pun of the ones I do get. Concerning the ones I've come up with, there aren't any puns, and the names don't fit the aesthetic of the others. Given all of this, I see two options:

Come up with thematically fitting names for the new species, and make sure they lead to puns.
Pick a new aesthetic target for the names, thereby obscuring the puns, and making the new names more coherent with the old names.

I'm trying to do the latter, and it's... not looking horrible, but I'm not quite gelling with it. I'm going to need to think about this for a few days, or something.

Anyway, I let things go late again, so I'm going to cut this here and get to bed.

Good night.

Diary 2024-05-01

2024-05-01T04:00:00-04:00

I had a busy-ish day at work today, so I ended up mostly just zoning out when I got home. Nothing to really talk about as a result. I'm not going to force things.

Good night.

Weekly Roundup 2024-04-30

2024-04-30T04:00:00-04:00

Wednesday: I described four projects, and sort of implied I was going to alternate between them. That, um, didn't happen.
Thursday: I tried to figure out how lunar orbital resonances do or do not relate to eclipse timing.
Friday: I ran the numbers, and they didn't work.
Saturday: I ran the numbers again, and they didn't work, but in a different way. In light of that, I decided to shift focus to more relevant aspects of the worldbuilding. First attempt didn't go great.
Sunday: Not much luck yet.
Monday: I went from not enough ideas to possibly too many. Then I decided to focus on research, and found out that ancient Rome had tower block apartments, so, like, how the heck do old-timey aesthetics even work?

Next week, I'm going to try to do a proper job continuing with this stuff, but we'll see how it goes.

Diary 2024-04-29

2024-04-29T04:00:00-04:00

Sapient species status: I may have too many now. Oops. I've got plenty more details to finish filling in there, but I've decided to try to shift focus to a few other areas. However...

It turns out that figuring out how to align the description of a world with "high-fantasy aesthetics" gets tricky when you decide to research things, and discover that, for example, the desire to shove as many poor renters as possible into tower-block housing predates modern construction techniques that make that "not a terrible idea" by... quite some time. So, like, there needs to be a solid reason in my mind why that wouldn't happen, or the extremely visible high-rise buildings need to somehow maintain aesthetic cohesion with the rest of the high-fantasy cities they'd be in.

Anyway, I don't have a quick answer to how to handle that right now, so I'm just going to think on it a bit. I want to wind down properly now.

Good night.

Diary 2024-04-28

2024-04-28T04:00:00-04:00

Not much luck yet in figuring out how I want different sapient species to work in this setting. Like, because of Reasons, I don't want there to be divine creation events, so I guess this setting (which isn't Earth: the moons are all wrong) has a long-term prehistory, in which humans evolved? In which case, are other sapient species a case of nearby offshoots or convergent evolution, or did weird stuff happen in the sprawling backstory?

(One thing that might tip the balance is the idea of the Ancient Evil getting resurrected, looking at one of the main characters, and going "Wait, what are you?")

I've also been trying to figure out how to tune the available magic and technology to get the setting aesthetics right, but I'd kind of like to know what kind of people are existing in this setting.

Anyway, I let things go late tonight because I was... doing... something? So I'll just get to bed now.

Good night.

Diary 2024-04-27

2024-04-27T04:00:00-04:00

I have finally worked through my desire to have the draconic months in my constructed world be "rigorously based on something" by the fact that I've found two different formulas, which give wildly different results for the actual Moon, neither of which match the actual value.

Therefore, nobody can prove that my numbers are wrong.

Anyway, with that off my chest, I can focus on the populations and history of the world. I've got some ideas here, though some of them, I think part of me is just being contrary. Like, I'm coming up with this for Fabula Ultima, which is a tabletop role-playing game, with definite ideas about character advancement and abilities and such. Some of the ideas I'm pondering ripping off from other people are almost certainly incompatible with the specifics. I've got to come up with character backgrounds that are somewhat closer to how humans interact with the world, if not necessarily close in terms of cognition.

I'll just keep that in mind for now. And get ready for bed.

Good night.

Diary 2024-04-26

2024-04-26T04:00:00-04:00

Okay, so, once I have the nodal precession rate of the moons calculated, I'll focus on stuff that's more pertinent to the story. But.

But.

But the numbers I'm getting aren't making any sense! Let's see if I can isolate the issue...

Before anything else, I have to check whether the numbers for the moon make sense to me. I keep on re-checking with the numbers for the Earth and the Moon, and the results keep on being off by four orders of magnitude. I'll try to work this out later, and focus on other stuff for now.

Good night.

Diary 2024-04-25

2024-04-25T04:00:00-04:00

Today: RPG setting. Not named yet, so no tag.

I'm procrastinating on the thematically important worldbuilding (What factions are there? What is the history of their conflict?) by trying to eke out just a little bit more rigor regarding the orbital dynamics of the moons.

I guess I should make the inciting incident to the story somehow revolve around a solar eclipse or something, or this is all going to feel pointless. (I mean, I can move the calendar and landmasses around however, so, um... look over there!)

I've got nothing else for tonight. Hopefully I get this out of my system tomorrow.

Good night.

Diary 2024-04-24

2024-04-24T04:00:00-04:00

All right, let's see what I have to say...

I've got a variety of attempts at worldbuilding under my belt, some of which I'm kind of torn between "I'd like to revisit this" and "I have Problems with the stuff I decided to position as Definitely Canon".

But aside from those, I've managed to accumulate four ideas that I've decided to try to develop in parallel, using a justification that kind of sounds like crop rotation for thoughts, if you squint:

The backstory of a novel; this mostly focuses on transposing historical trends and ideas into a fantastical setting, for the purpose of really straightforward commentary
A series of extremely short stories in a setting that I'm deliberately leaving vague outside of what I can extrapolate from the text of the stories
A speculative evolution project based on fantastical principles
"This tabletop RPG has guidance for worldbuilding the setting, and I'm just going to read it, I'm not going to—okay, let's go"

The basic idea is that each of these hopefully scratch a different kind of itch. We'll see how that works out... For now, I really need to get to bed.

Good night.

Weekly Roundup 2024-04-23

2024-04-23T04:00:00-04:00

Wednesday: "Boy, it sure would be nice to have a free offline kanban app"
Thursday: "How about using Flet?"
Friday: "People care about version pinning via lockfiles, but it turns out that for this project, I can afford not to."
Saturday: "I'm understanding Automerge slightly better than I was before..."
Sunday: "Understanding it still better..."
Monday: "Ah, I caught up to stuff that Automerge's developers knew months/years ago; it's not ready for what I want to do with it here."

Next week, maybe I'll write up what else I was working on last week, maybe I'll focus on something else.

Coding 2024-04-22

2024-04-22T04:00:00-04:00

The backend prototype is about done, and thinking about it, I am fairly confident that the design is not sound. If I'm reading the Automerge documentation correctly, "removing an value from an object" and "putting that value in another object" would be considered as distinct actions for the purpose of reconciling divergent edit histories.

So, if I'm understanding this right, then the way I implemented the prototype allows for a situation where "a card is moved from position A to position B" and "a card is moved from position A to position C", merges as "the card is simultaneously present at position B and position C".

I should actually test this, but I'm not sure how the current implementation could do what I want. I'm trying to figure out what "what I want" would look like. Somehow representing a rotation of a sub-list as an atomic operation... I'll let that idea percolate, but unless I come up with something, it's probably not worth building this on top of Automerge.

...

Having done some research, another option is that I could simply wait, since the problem I hit is not something super-novel or surprising, and Automerge's maintainers are already working on a solution.

Once that's in place, there are a few other things I'm still not sure how best to handle, but they seem like smaller problems, and if I'm wrong, people are probably working on them.

Let's see about putting this down for now, and picking up something else. I've actually been working on something else for the past week, but it's been even more of a pain. More on that later.

Good night.

Coding 2024-04-21

2024-04-21T04:00:00-04:00

I spent a lot of today messing around with Automerge and figuring out whether I've been doing things wrong.

I've implemented a bunch of functionality on the backend, but I'm still missing some key bits. At this point, it's possible to create and delete all of the relevant elements, and do most of the actions related to rearranging them. What's missing is the ability to update the various text fields that are in there.

I haven't tested any of this code, so it's probably going to do something hilarious the first time I try to put it all together.

It's late enough that I shouldn't try to do anything else, so, time for bed.

Good night.

Coding 2024-04-20

2024-04-20T04:00:00-04:00

Okay, I have attempted some prototyping. I now have a slightly better idea of how I want to be using the CRDT library I'm using, Automerge. For the specific idea I have in mind here of "here are cards, and they can be moved around", I think I need to be focusing a lot on arenas. This is because I don't see a way to move an object around in Automerge, so I think whatever gets moved around has to be a scalar identifier.

I'm not totally certain about how I want to be representing the objects in these arenas. Like I probably want to be using the arena index as much as possible and only resolving it to an object id at the latest possible moment, but right now that's just a gut feeling, and I think to get any closer to understanding the situation, I need to take the pile of API methods I've noted that I'll need, and consider the signatures I want for them, which may be constrained in some manner by implementation details.

We'll see what I can manage tomorrow.

Good night.

Coding 2024-04-19

2024-04-19T04:00:00-04:00

Current "pretend I'm working on the kanban app but I'm actually not" status:

"Hmm, since this is an application, it sure would make sense to have some kind of lockfile. I wonder if any of the choices for that right now don't somehow bother me."

...

Oof, big "everyone complains about / nobody uses" energy in the current state of things. mousebender sounds nice and targeted in its focus, but it appears to be early days yet. Aside from that, we've got:

Poetry
PDM
uv
pip-tools
Pipenv

Doing a bit of research, I think I've found stuff to be unhappy about with all of them, so let's take a closer look at Flet's build process for Android and see if I can find a pragmatic choice for this particular scenario.

Okay, so, I don't know what anyone more knowledgeable about the relevant parts of the Python ecosystem would think of the statement "p4a is kind of like nuitka", but I'm just going to breeze past that so I have some kind of baseline idea of how this all works. Flet requires the patch version of p4a to be pinned lower than the current CPython release, which is a little spicy... However, as far as guaranteeing functionality, I should be "okay" if I match the minor version in my development environment. I'm not seeing a way to directly specify version constraints as part of p4a create --requirements; the documentation only mentions names. It is possible to specify pins through the setuptools integration, but this just seems cursed...

Wait, hold on, do I need to be packaging all dependencies, or just compiled ones? The Flet documentation makes it sound like any packages that don't need a recipe can be installed automatically during the flet build step, I think?

However, there's an additional wrinkle: the CRDT library I want to try to use here is off in pre-release land right now, so pinning its version or somehow telling p4a to take development versions is necessary in order to make it... work.

Remembering how I got started on all of this, it is... helpful... to realize that I don't have any other non-Flet dependencies, so locking is kind of irrelevant.

Ultimately, it makes better sense right now, I think, to make sure I can get all of this functionality working at all, on the desktop, and shelve the whole "get it nicely on my phone" idea for now. (Unless I can turn it into a static site that's installable as a PWA, in which case, that's great. Seems a little improbable, though.)

So, I guess the right course of action right now is to... Start actually prototyping stuff, and try to get a feel for these libraries.

Okay, I'll get on that. Tomorrow.

Good night.

Coding 2024-04-18

2024-04-18T04:00:00-04:00

Hm.

I got all set up in Flet, and after a bit of struggle, managed to preview my app on my phone, but as I look at and tinker with the preview, I'm skeptical that the app preview is telling me what I really want to know. Like, if using the wrong version of a package causes import errors to pop up on my laptop instead of my phone, then that doesn't tell me anything about whether the Python packages I want to use can be cleanly packaged into an apk, right? I guess the point is to make sure that the UI isn't just totally unusable on my phone.

...

And from the radical step of reading the documentation, I see that compiled packages are a problem, they did anticipate it, there is a solution, it's not turnkey for what I have in mind, but it doesn't look like bashing together something for my own use is a fundamentally unreasonable proposition. I won't have this all ready for when I want it all ready, but it seems likely I can get something sort of workable soonish, at least as far as Flet goes.

So, that's some brief investigation, and I think I want to get off my laptop for tonight, so I'll focus on paper prototyping or reading.

...

I did things in the wrong order. So, that's some good barebones prototyping done, and a bit of research on Flet. The one thing I'm concerned about is the whole "Oh, we layered imperative on top of Flutter's declarative, so you don't need to worry your silly little head about defining the UI as a function of the state" thing. Like, I look at that, and I've got the twin dæmons of Spite and Pragmatism shaking hands and smirking over the concept of writing a function to regenerate the UI from scratch every time the state of the underlying data changes. Like, I'm going to have enough trouble getting the drag-and-drop behavior (which I kind of need) to propagate down to the data, I don't want to be manually optimizing the updates to the representation of that data.

Anyway, really should rest my eyes now.

Good night.

Coding 2024-04-17

2024-04-17T04:00:00-04:00

Over the weekend, I ended up wanting to make a simple application (which got somewhat less simple as I iterated on my requirements), and I investigated a few options.

Filed a bug report as a result of this of which the following two things are true:

The report ended up being pretty bad, and this is my fault
I'm still not interested in using the project in question for now

(Aside: I installed a bunch of shells on both my computers, and set up some probably unnecessarily complicated Lua code to switch between them. I haven't figured out how to get it to work with the terminal emulator on my laptop though... This is only important because it's what I was doing most of the day, aside from my job.)

Anyway, the project I was looking into this for is to have an app that does the following:

Kanban board
Local files
Where I can find them

Lots of kanban board apps are like "just spin up the server in Docker", and... no.

Here's how things look for the details I've thought about:

Must work on linux
Bonus if it works on Android
Needs some kind of save format that is better than pickle
CRDTs would be nice because then I could try to handle merging changes from different devices
Otherwise, human-readable
Handle format versioning somehow

The tradeoff with "hey, what if I use CRDTs" is, I'm not sure how the usability of current CRDT libraries is. Also, that might interfere somehow with the whole "what if I could also use it on my phone?" thing.

So, I think the thing to do is to put together a toy CRDT-based app and try and figure out how to run it in the simulator, or whatever.

Anyway, I desperately need to get to bed soon.

Good night.

Weekly Roundup 2024-04-16

2024-04-16T04:00:00-04:00

Wednesday: I made enough progress on MOTR that I decided it was time for another break.
Thursday: I picked up another project: making small improvements to someone else's code.
Friday: That all worked out well enough.
Saturday: Nothing much.
Sunday: Nothing much.
Monday: Nothing much.

Next week, I've got some other projects I want to look into. We'll see how things go.

Diary 2024-04-15

2024-04-15T04:00:00-04:00

We're back. It's late. I'm tired.

I can't work my way up to trying to write anything, so that's that.

Good night.

Diary 2024-04-14

2024-04-14T04:00:00-04:00

On the road. Gaming.

My energy was directed at a bunch of stuff today, none of which really makes sense to me to write about, so, eh.

I'm going to wind down now.

Good night.

Diary 2024-04-13

2024-04-13T04:00:00-04:00

Okay, let's see. I decided to take things easy today, and I should for the next few days as well.

I don't have anything in mind to talk about right now. Let's sleep.

Good night.

Coding 2024-04-12

2024-04-12T04:00:00-04:00

Hm. It turns out that updating a Python project to current standards is pretty easy, if there isn't anything too exciting going on in the setup.py file. And the actual problems that were the whole motivation were pretty simple to fix.

At this point, the bulk of the work is going to be discussing the changes I made. I'm super not up for that right now. I just want to lie down indefinitely, which unfortunately does not gel with my plans for tomorrow or the weekend. Anyway, I'd better get what rest I can manage.

Good night.

Coding 2024-04-11

2024-04-11T04:00:00-04:00

I got a little bit of writing done today, but the big thing I started working on right now is preparing to update someone else's project to work a bit better in a few ways. Getting this right means putting aside my own preferences about how the project "should work", and researching only the tools they're already using (except also refreshing my memory on tox, because I cannot in good conscience make anyone else use MOTR yet). I think I've got enough notes taken to get to work later, but for now I should have gotten to bed like half an hour ago, so, um... bye.

Good night.

Coding 2024-04-10

2024-04-10T04:00:00-04:00

Okay, I've got some changes I made, and now I'm feeling better about some of the names in MOTR.

At this point, it remains to be seen whether there are more changes to make. I could keep on this, but I feel like it makes sense to take a break for a few days and come back to this totally fresh, instead of mostly fresh, if that makes any sense.

It's interesting that making these updates only involved changing a single file; that means I should be able to make further updates later if I feel the need, even after release.

My last thought on this for now: I still haven't decided if I want to go through with untangling artifact from matrix. It's like, that seems like it would be nice, but I'm not sure how exactly to do it without making some really messed-up names.

...

One possibility would be to simply widen the type, since the mapping type is, I think, a strict subset of other mapping types defined without reference to artifact. I'll turn that idea over in my head for a bit.

For now, I want to get ready for bed.

Good night.

Weekly Roundup 2024-04-09

2024-04-09T04:00:00-04:00

Wednesday: I considered how I want to rearrange MOTR.
Thursday: I made some weird prototypes, and then took a look at the more grab-bag-y parts of MOTR's code.
Friday: I found some specific weird bits of the code and started thinking about them.
Saturday: I did what I was thinking about, and then discovered that MOTR's validation suite had stopped passing because various dependencies updated in the last three months.
Sunday: I made the required updates, and then found another change that I wanted to make to the code.
Monday: I made the change, and took another look at the most obviously bad names.

Next week, I'll try to keep this up, and maybe do something else as well.

Coding 2024-04-08

2024-04-08T04:00:00-04:00

Okay, I made the changes I was thinking of earlier, to give PathStr its own module, and move around some TypeVar definitions. That went well enough, though I did get caught off-guard by a few imports I couldn't remove, or move to where I was thinking of moving them.

In any case, my focus moved back around to fixing up the names in parametric_command. I'm still trying to rename that one protocol; maybe if I try to explain what it's for, I'll get some sudden insight I can apply later.

So, there's a class called ParametricCommand. It can either represent a set of arguments to a Python module invoked via python -m, or arguments to an executable command. This is a helpful distinction because some Python tools offer a way to run that is "like python -m but". One of its attributes is either one of those "pass arguments to python -m" instances, or a private wrapper class around a "matrix.SelectionDirectory", which is sort of a juiced-up/stripped-down matrix.Matrix. In this case, it holds information about how to determine the path to the installer script for the virtual environment the command executes in.

Ultimately, this attribute determines stuff about the virtual environment path, the path generated reports get sent to, and probably something else, but I'm not sure.

Food for thought. Time for bed.

Good night.

Coding 2024-04-07

2024-04-07T04:00:00-04:00

Okay, I fixed stuff up in MOTR:

I changed the @pytest.fixture() lines to @pytest.fixture.
I switched the ExceptionGroup checking to using the trio.testing version for now, and got rid of the exceptiongroup package altogether.

At this point, one thing I need to do is really consider whether I want to go through with the idea of decoupling matrix from artifact through parameterization. This is a tough call to make right now, because the purpose of the input field on the axes is purely as part of the API surface, so whatever I do with it has to make sense to someone working with a motrfile. (Unless I somehow come up with yet another layer of API to lather onto this.)

One possibility would be to write a helper function in the artifact module...

It may be best to simply try to note down all of the relevant relations, and just work out how they all interact.

...

Poking at the code, I think the next big improvement I can make is to have a dedicated module for the PathStr alias. The artifact module also defines some type variables that really should get defined in different modules.

Making those changes would do good things, I think. I'll try and get to work on them tomorrow. For now, I want to get to sleep.

Good night.

Coding 2024-04-06

2024-04-06T04:00:00-04:00

I thought really hard, and I came up with the course of action I want for artifact and parametric_command in MOTR, and I executed on it, and then that worked, near as I can tell, but then the MOTR run failed for unrelated reasons, and around the same time, my laptop battery dipped ridiculously low. So, I'm going to try to get this all working tomorrow.

The course of action I chose was to inline the function body where it was used in artifact, and make the top-level function a private staticmethod. This all basically makes sense, but I don't want to think about it any more tonight. I'm going to read a bit and get to bed.

Good night.

Coding 2024-04-05

2024-04-05T04:00:00-04:00

I continued to pick at the scab that is the knot of cyclic dependencies surrounding the matrix module, and I noticed something that I'm not quite sure what to make of it. Let me see if I can explain what's up...

There are four modules that appear to matter here:

invocation_environment, which defines a class that bundles expressive command-line specifications with the installer metadata required to convert the specifications to a concrete form
matrix, which defines a bunch of classes, which are responsible for handling axes of variation and instantiating every combination of the relevant axes
parametric_command, which models command-line specifications at the highest level of abstraction currently used in the project
artifact, which handles the details of coordinating the prerequisites and outputs of command-line invocations
dependent_mapping, which is used all over the higher levels of the codebase, but is particularly relevant for the implementation of the matrix classes

Five modules.

invocation_environment has a transitive dependency on dependent_mapping, but doesn't otherwise use any of the other mentioned modules. Both parametric_command and artifact have a runtime dependency on it.

matrix and artifact have a runtime dependency on dependent_mapping.

The other major chain is that, at runtime, artifact depends on both parametric_command and matrix, and parametric_command depends on matrix, and there are additional type-time dependencies, frpm parametric_command to artifact, and matrix to artifact.

The two links that really have my attention right now, for different reasons, are the type-time dependency from matrix to artifact, and the runtime dependency from artifact to parametric_command. This is because breaking the type-time dependency is desirable aesthetically, but I think the natural way to do so would introduce a runtime dependency from parametric_command to artifact, which would be a cycle, unless I can break the existing runtime dependency, and thinking about this highlights the fact that the existing runtime dependency is... strange. artifact just needs one top-level function from parametric_command, and surely this function is doing something that would make it hard to move, right?

Right?

Pause for emphasis.

Well, invocation_environment (remember invocation_environment?) is the only module needed to implement the function. In fact, it's somehow worse than that makes it sound. This function is basically a restricted version of a classmethod from that module.

So, like, should I be trying to break the dependency? Thinking about it, I suppose it comes down to figuring out the semantics. It's a tough hypothetical, because this code seems so... elemental... but I need to figure out which module needs to change if I decide that the behavior relied on by artifact needs to change. The options I see now are:

Inline the classmethod call in artifact
Move the function definition to invocation_environment
Move the function definition to artifact, and reverse the direction of the runtime dependency

I've just put these choices together and I haven't had time to consider them all yet. Hopefully I'll figure something out later. For now, I want to get to sleep.

Good night.

Coding 2024-04-04

2024-04-04T04:00:00-04:00

I've got some bad news about this one, folks.

Aw, come on!

Show them what you decided to work on.

But—

:) (glitchy)

Show them!

Fine...

from __future__ import annotations

import abc
import typing


class Combinable(typing.Protocol):
    """Protocol for classes whose instances can be combined."""

    @abc.abstractmethod
    def combine(self: T, __other: T) -> T:
        """Combine two instances of the same type."""
        return self.combine(__other)


T = typing.TypeVar("T", bound=Combinable)

I can't help but notice that you're proposing to replace a single lambda with two lines of code.

It's unaesthetic. Besides, I haven't actually touched any code yet. That up there is the prototype. This whole exchange was a rhetorical framework to give me an excuse to write the code.

Touché. Regardless, it is impressive and horrifying how much nonsense mypy appears to be prepared to accept from you.

Yeah! It's all like "So, you specifically want a union to be considered to implement a protocol? Yeah sure I'll add that to my worldview."

The point is. What actually needs effort put into it is isolating modules like this so it's possible to find common themes in what remains. Updating the code in any way seems quite beside the point at this juncture.

All right, I'll try to follow that advice, and not solely because I just got tired of typing this post in two voices.

Looking for modules without many dependencies. Let's see...

artifact: definitely not
box: an odd case; it only relies on some modules that are in this list for sure, but its actual usage is pretty specific to the matrix concept
combinable: for sure
entry: doesn't import anything, but I doubt anyone else would want to rely on it, so what would the value be in breaking it out further?
environment: somehow tangled up in "core" logic, so presumably no
executable: I mean, it would be possible to make a fully generic version of the data type and then simply specialize it, but that's the kind of code someone writes to win a really unimportant argument
installer: the key thing here is the reliance on matrix; which sounds like it should be generally useful, but actually isn't
invocation_environment: builds on a bunch of "core" code, which presumably somehow explains how this file manages to feel "short" at 200+ lines
label: I would happily publish this in a standalone package, and nobody would use it; you may not like it, but this is what peak performance looks like
matrix: on close inspection and consideration, this code is actually really close to being some kind of standalone thing; actually pushing it the rest of the way would involve some extra type variables, which this module already has in abundance, and also the API surface is currently not remotely stable
not_output: not worth thinking about
parametric_command: confusing and unstable
path_with: if I can get combinable out, I'd love to have this follow; it relies on various bits of code that don't specifically pertain to MOTR
program: meh
runner: heavy usage of "core" functionality
selection: it can go where box goes; which is, for now, right where it is
bonus modules: dependent_mapping, saltate and validators, which just make sense to give their own packages

All in all, this only gives us a few packages with license to ignore them:

combinable
dependent_mapping
label
path_with
saltate
validators

The really valuable ones to remove are dependent_mapping and saltate, because they slow down testing.

As far as organizing things, I'd like to pull out the matrix ecosystem into its own submodule at least, but the circular dependency between matrix and artifact, as well as artifact's various "core" dependencies makes me not like that. Perhaps if I could find a non-obnoxious way to break the dependency.

Anyway, one thing that's getting hammered home from inspecting the usages of the various modules is that I really should move cmd.PathStr into some kind of generally usable module, because many modules import cmd only for PathStr.

I think, ultimately, I can only take typing up notes so far. I'm going to have to print out a module listing and scribble on it.

Not right now, though. I think I taxed my eyes a bit writing this, so I'm going to get away from my laptop.

Good night.

Coding 2024-04-03

2024-04-03T04:00:00-04:00

I'm not really focusing on coding right now, but I want to do some planning, at least.

In my review of the code recently, I re-confirmed that there are various modules with no dependencies on the rest of the code, so it should be workable to separate those out. The hope is that doing so will make it easier to determine common themes in what remains.

Actually handling all of this will probably be a bit confusing, because the ideal form of the code exists in a sort of hypothetical octopus merge of multiple development topics and some changes that I haven't made yet because I haven't figured out what they actually are, so that's going to be interesting to notate.

Anyway, I've let this go long enough tonight. Longer, even. I should get to bed.

Good night.

Weekly Roundup 2024-04-02

2024-04-02T04:00:00-04:00

Wednesday: I did some planning related to Tsetatsatal, and thought of some nice-to-haves for the interactive IPA stuff.
Thursday: I cut down on screen time at the exact moment I would have been writing a blog post.
Friday: An invisible timer somewhere deep in my psyche went off, and I started looking at MOTR again.
Saturday: I came up with some weird code golf stuff for MOTR that would probably be a bad idea.
Sunday: Video Games
Monday: Books

Next week, I'll see if I still have the urge to work on MOTR. Also, I might as well tinker on Tsetatsatal stuff.

Diary 2024-04-01

2024-04-01T04:00:00-04:00

And, we're back.

It would be nice if I had something to say, but, eh. I was reading books and thinking about those books. Tomorrow, I'll keep on winding down, and then try to ramp things back up.

But for now, I need to relax.

Good night.

Diary 2024-03-31

2024-03-31T04:00:00-04:00

Guess what, it's another travel weekend. I did poke a little at MOTR, but I'd like to get back home before I try to do much with it.

Besides that, I read a little and played video games a bunch. Anyone wants to recommend Baba Is You or Patrick's Parabox levels, poke me on Mastodon or Cohost. (Or other games, but I have a lot of games...)

Anyway, I should get to bed before it gets too much later.

Good night.

Coding 2024-03-30

2024-03-30T04:00:00-04:00

I'm looking at the code in MOTR to try to get a sense of how it all fits together, and naturally this means I've got a niggling voice in my head going, "You know, you could remove a lot of this boilerplate by writing something like..."

def delegate(
    method: typing.Callable[typing.Concatenate[T1, P], T1]
) -> typing.Callable[typing.Concatenate[T2, P], T2]:
    wrapped_class: type[T1] = # Some horrible nonsense I don't want to go into.
    @functools.wraps(method)
    def wrapper(self: T2, /, *args: P.args, **kwargs: P.kwargs) -> T2:
        for name, attr in attrs.asdict(self, recurse=False).items():
            if isinstance(attr, wrapped_class):
                return attrs.evolve(
                    self, **{name: method(attr, *args, **kwargs)}
                )
        raise TypeError(f"Did not find a field of type {wrapped_class}")
    return wrapper

I'm pretty skeptical that mypy will, accept this nonsense.

...

I just got it to accept broadly equivalent dataclasses code.

Oh dear, this could work.

And the other boilerplate! I could write a class decorator to convert a smattering of metadata into a standardized method.

No! Back away from the edge! Mypy still doesn't handle class decorators that change the type of the class. To get it to typecheck, you'd probably need to write metaclass or something.

Also, this idea is completely deranged.

Ultimately, while I could try to golf things down a little, it makes more sense to get things documented and try to improve the names. Annoyingly, this stuff is calling for some pretty intense precision in names, and I feel like the pre-existing concepts I'm trying to track tend to be referred to a bit more flexibly. I'll have to come back to this when I'm a bit better rested.

And I should get to work on getting rested.

Good night.

Diary 2024-03-29

2024-03-29T04:00:00-04:00

Ugh. My attention is drifting back towards trying to get MOTR working, and that means I need to remember what the heck I was doing. hg log informs me I spent over a month in a topic called "risky-renames" before putting the project down for three months. Poking at it, I can't suss out the larger trend yet.

It looks like, if I want to get anywhere with this, I need to start over with documenting the different names in the code and what I think of them. And my last entry claims I'd "finished" something. I can't tell if there are other issues besides the very obvious, so I'm going to do some surface-level analysis, then try and take some time away from my laptop.

There are three topics I'm juggling right now:

risky-renames
87-test-coverage
39-documentation

Once risky-renames is ready, it gets integrated back into 87-test-coverage, and I merge the updates to 39-documentation. At that point, I go back to 87-test-coverage, and get back to work on test coverage. That bit's pretty self-explanatory.

So, ultimately, I just need to get risky-renames into a state where I'm happy with it, and then I can move on. (I'm not happy with it right now. Some of these names are bad.)

Anyway, time to try and clear my head.

Good night.

Diary 2024-03-28

2024-03-28T04:00:00-04:00

I'm not sure if I didn't get enough sleep or if it's something in the air, or what, but I'm feeling awful right now, so I'm going to step trying to do stuff on my computer, and start trying to do stuff on my e-reader.

Good night.

Coding 2024-03-27

2024-03-27T04:00:00-04:00

I've got some tables I want to make for Tsetatsatal, and they could potentially get very complicated, but first I want to figure out how much complication I need.

Consonants, places of articulation:

labial, possibly considering bilabial separately from labiodental
alveolar
palatal
velar

Consonants, manners of articulation:

nasal
plosive
fricative
affricate
approximant
tap/flap
lateral approximant

I'm going to have to figure out whether I want to add, remove, consolidate, or relocate stuff to deal with how sparse the chart is.

Vowels are pretty straightforward: open, close-mid, close, and back rounded, front rounded, and front unrounded (not open (if I'm not confused here)).

My want for the code I was planning to write is to support stuff like the weird set of spans that the Received Pronunciation vowel chart has on Wikipedia, but I can make do without that for both of these tables, so I might as well get on that.

...

Hm, as I contemplate the prospect of nasalizing all of these vowels, I realize that it would be nice to somehow apply diacritics to a full paradigm. Like if I put them into the field whitespace-separated, and it just maps the diacritic over them.

Okay, let's see. I've copied a bunch of phones out, and I can try to fit them into whatever table I come up with, but it's late and I want to just work on that tomorrow or so.

Good night.

Weekly Roundup 2024-03-26

2024-03-26T04:00:00-04:00

Wednesday: I laid out the pulmonic consonants.
Thursday: I laid out... something.
Friday: I started trying to figure out what needs to be combining characters. The answer is "a lot of things".
Saturday: I got everything that wasn't those things basically squared away.
Sunday: I gave some thought to how to lay out the fancier interfaces I'm going to want, and read marimo's documentation a little more closely.
Monday: I got enough of the IPA stuff implemented for my current purposes, so I decided to leave the remainder as TODOs for now.

Next week, I'm going to try to implement some really weird-looking helper code, then actually touch Tsetatsal's documentation, maybe do some other writing as well. Hm. Maybe Tsetatsal could be part of that setting. Can't prove it's not.

Coding 2024-03-25

2024-03-25T04:00:00-04:00

Okay, I've got a bunch of diacritics implemented. At this point, I'm missing tones and contours, and general superscripts. I don't need those for Tsetatsatal, so when I pick things back up, I'm going to work on the chart stuff. First laying the groundwork with some helper code, then making the consonant and vowel charts for Tsetatsatal.

I'm kind of out of it after putting those diacritics together, so I'm going to wrap things up for now and get ready for bed.

Good night.

Coding 2024-03-24

2024-03-24T04:00:00-04:00

I mostly took things easy today, but I did take some notes on how I want to handle combining characters etc in IPA. It's not quite there yet.

Here's the deal: a bunch of the ideas I have involve multiple text fields feeding into the output, with different purposes. It more-or-less makes sense to put these text fields right next to each other, but then I'm not sure how to indicate the different purposes of them. Like, maybe I should just stack them vertically to make room? But putting them horizontally best indicates what's supposed to be going on with them. Oh, wait, placeholder text. There we go.

I really should learn marimo properly sometime, huh?

We'll see if I'm up for trying out these interfaces tomorrow; I was dealing with sensitive eyes all day, but maybe I'll do better tomorrow. For now, I want to put my laptop away.

Good night.

Coding 2024-03-23

2024-03-23T04:00:00-04:00

Okay, I believe I've coded in every part of the IPA that isn't some kind of modifier or combiner. That's probably going to end up having been the easy part, because there are so many diacritics and such in the IPA. I'm going to have to figure out how to handle all of the details of that.

For now, I want to get off my computer and go read ebooks.

Good night.

Coding 2024-03-22

2024-03-22T04:00:00-04:00

I'm doing prep work for the helper code: specifically, figuring out which of the remaining symbols are some kind of combining character. The answer is: more than I thought, and I already thought "most of them".

The prep work also convinced me to reorganize the existing work a bunch, so that's... cool...

Anyway, I drastically need to rest my eyes, so I'm going to wrap up early tonight.

Good night.

Coding 2024-03-21

2024-03-21T04:00:00-04:00

I've coded in a bit more of the IPA stuff. I should take some time and figure out which of the things I've yet to add, should be added as diacritics.

It's late and I have nothing else to talk about.

Good night.

Coding 2024-03-20

2024-03-20T04:00:00-04:00

Well, I've got the pulmonic consonants in, and unfortunately, I fried myself a little by laying out the HTML for the table totally manually. I'm going to have to give myself some time to cool off, and then I can work on copying more symbols in, half-assing the vowel trapezoid, and then figuring out how I want to handle the diacritics.

It seems I have nothing else to say on this, so I'm going to get to bed.

Good night.

Weekly Roundup 2024-03-19

2024-03-19T04:00:00-04:00

Wednesday: I thought some about conlangs that have failed in their goals, or at least not succeeded persuasively. I had some ideas about what "went wrong", but I'm not sure if those ideas are more "insight" or "assuming my experiences are universal".
Thursday: I thought my plan for Tsetatsatal would "get less away from me" than my previous attempts. Ahahahahaha.
Friday: I made some progress on Tsetatsatal, and then fled the screens.
Saturday: I don't remember what I did.
Sunday: More progress on Tsetatsatal.
Monday: I tried to get some idea of what would work for IPA characters, and mostly got ideas of what wouldn't work.

Next week, I'm going to try to put together basic stuff for the IPA, then helpers for phonetic charts, then get back to Tsetatsatal.

Diary 2024-03-18

2024-03-18T04:00:00-04:00

Starting this post early because I had stuff that can go in it on my mind...

I had a look at just a few phonology sections on Wikipedia to refresh my memory, and from looking at them it looks clear to me that "just subset the IPA charts" is not going to work well. My inclination is to say "people can come up with whatever rows and columns they like, although I'd like to provide some sensible defaults". And "it should be possible to nest heading cells in verious ways, and I'm not confident right now that I can figure out the proper manifestation of the nesting automatically". What goes in cells should be relatively freeform, but I'd like the ability to choose between "centered, left-justified, right-justified, some text left- and some text right- justified" on a per-cell basis. As far as IPA symbols, that should be entirely freeform, but it would be good to have some helpers for stuff like aspiration and devoicing.

And those helpers look to be a minor nightmare to put together, because the IPA symbols have a lot of stuff going on, in terms of unicode representation. And Wikipedia isn't helping, with the frankly bizarre styling it inflicted on the table I'd like to use, which seems to make it entirely unsuitable for printing. (Also, there's something going on with the click consonants that doesn't look right. I don't have plans for click consonants in Tsetatsatal, but it would offend my sensibilities if I ignored them or got them wrong. That said, it looks like so much of the details of click phonemes is represented through diacritics, that it looks to me like the sensible course of action initially would be to have a separate helper just for getting the IPA character for the place of articulation, and punt on filling in the rest of the details for now.)

Those digressions aside, I think I have to acknowledge that, while it would be cool to build up some of the IPA consonant representations in a sort of refinement process, it makes much more sense to create a hardcoded lookup table. Sadly, even this isn't totally straightforward, because there's a lot of stuff going on in the various IPA tables I'm looking at, and they also don't agree with each other in some significant ways. Stuff like "do we call out dental plosives as distinct from alveolar plosives?" or "so what should the deal with sibilants be?" and especially "so what does it mean in these charts when the line between cells ends halfway between?" At this stage, it seems like it would make more sense to simply embed the IPA chart in the supporting cells and be all "copy out the symbols that you want".

Plausible other things to work with would be stuff like "combining mark/spacer applier" or "superscript converter".

Looking at what I can find easily, there are a few issues, which I'll need to take some time to remedy. I'll get to it in a few days, hopefully. For now, I want to get off my computer.

Good night.

Diary 2024-03-17

2024-03-17T04:00:00-04:00

Ugh, I was trying to fix some stuff I messed up on my Deck, and it turns out I was working with outdated directions, so that wasted a bunch of time. I'll come back to it later, because I don't want to keep thinking about it right now.

I made some progress on the conlang stuff, but mostly on the code side. I think I need to add a bunch of placeholder entries so I have some idea of the scope of what needs to get filled in. I'll try that now.

...

Mm, that's a bunch of work to do later...

Thinking over how to move forward, I think I should be focusing on phoneme charts. That'll help with the possibility of missing phonemes in words.

Anyway, I'll think about that (it's probably going to end up with functions to generate the HTML explicitly), but right now, I'm going to switch gears for a bit. What I've got in mind is not too interesting to talk about; it's just something simple that I want done.

Good night.

Diary 2024-03-16

2024-03-16T04:00:00-04:00

Okay, I didn't get much done today, so I don't really have anything to talk about. I'll just try to get something interesting to talk about tomorrow.

Good night.

Coding 2024-03-15

2024-03-15T04:00:00-04:00

So, the good news is that I have put together some prototype code for dealing with conlang stuff in marimo (the stuff I'm doing works as a more general feature request, which I've filed and I intend to write up a recipe for replicating what I'm doing), and I've started adding lexicon entries. It feels decent enough working with all of this, so that's all good.

The bad news is that my head has hurt all day, and I should get away from screens again.

Good night.

Diary 2024-03-14

2024-03-14T04:00:00-04:00

Well, I didn't think any more about the stuff I was thinking about yesterday. Instead, I decided to start writing stuff down about the language I currently have in mind.

The basic thing I'm trying to work out for myself is how far I can push the concept of stative verbs, in the direction I'm interested in.

I'm also looking into writing up a conlang grammar in marimo, on the supposition that maybe that will get less away from me than trying to write a grammar in Sphinx. It might have gotten away from me a little, because I'm trying to work out how to handle stuff like phonetics charts and the lexicon. Probably, I should just bite the bullet and do stuff by hand at first, then convert it all into stuff upstream. But not right now, because I was messing around with other stuff earlier. For now, I need to try again to rest.

Good night.

Diary 2024-03-13

2024-03-13T04:00:00-04:00

I'm feeling really tired right now, going to have to deal with that somehow.

I was thinking some today about the history of conlangs, and the way that the craft of conlanging has been entangled to varying degrees with utopian thinking. I personally haven't had to deal with this any (except for from myself once like twenty years ago, when I was sure I'd cracked the code to representing and manipulating concepts in a consistent framework with a simple phonetic representation...), so I'm thinking about this in terms of its historical legacy.

I don't know if my thoughts are well-developed enough for me to go into much more detail without accidentally offending someone (I'd like it to be on purpose), but that is what I've been thinking about a lot. And questions like...

Wait, how are we defining success?
Will a new language meaningfully advance that goal?
How have we determined what is wrong?
How reliable are those methods of determination?

Sadly, it turns out that linguistics ability is not a substitute for project management. Lots of things aren't, like programming ability.

Ugh, I'd like to end this post on a more definite note, but I'm feeling incredibly tired. Like, all I can say for how I feel right now is that it is possible to feel even more tired than this, which is not an impressive bar to clear.

Good night.

Weekly Roundup 2024-03-12

2024-03-12T04:00:00-04:00

Wednesday: I finished work on the conversion script, and rattled off a list of bugs I had to fix that probably made the script sound extremely fake. Like, I had problems with a custom parser, xpath handling, and unpickling? Ridiculous.
Thursday: I had a rough day.
Friday: I had another rough day.
Saturday: I had a less rough day.
Sunday: I tried to get back into writing, and didn't get too far.
Monday: I was very sore, and not sure what I felt like doing.

Next week, maybe I'll get back to writing, maybe I'll mess around with other stuff. We'll see.

Diary 2024-03-11

2024-03-11T04:00:00-04:00

Uugh. I have been sore all day.

As a result, I didn't really focus on much. While I've been trying to get back to the writing I was doing before, I feel like I'm kind of at the mercy of my whims, and my whims are inclined towards conlang stuff right now. I've got some ideas I'm messing with, but nailing stuff down there is, um, more writing, so I figured the sensible thing to look into would be reading about other conlangs that strike my interest.

Rather than drag out this entry any longer, I'm going to get off the laptop and try getting back to that.

Good night.

Diary 2024-03-10

2024-03-10T05:00:00-04:00

Oh hey, it's the yearly "when is the right day to adjust the timestamps" moment. I think I switch them tomorrow.

Anyway, I'm trying to ramp things up at a reasonable pace for now. For writing, I've got "update the build pipeline" and "copy everything into the outline" as blocking things. I tried to get stuff into the outline, and I had trouble with how un-fleshed-out the beginning is; I'll take another stab at it later, but for now I'm working on transferring setting notes.

Right now, I want to get to sleep and deal with the time change.

Good night.

Diary 2024-03-09

2024-03-09T05:00:00-05:00

Wow, I feel not-great at the moment. At least I don't have a headache.

My current hope is that this clears up by morning, and I can focus on destressing.

But right now, bed sounds good.

Good night.

Diary 2024-03-08

2024-03-08T05:00:00-05:00

I'm currently hoping that tomorrow, I'll be able to take care of the stuff I've been over-focusing on this last week. Then coast through the weekend.

I don't have anything else I want to talk about right now.

Good night.

Diary 2024-03-07

2024-03-07T05:00:00-05:00

Well. I currently lack the focus to do much, so I'm just going to publish this and get off my laptop to try and let my eyes recover.

Got to stop staring at this like I might think of something else to add.

Good night.

Coding 2024-03-06

2024-03-06T05:00:00-05:00

To all indications, the conversion script works now. Tally of bugs:

I needed to rework the code to extract node metadata, because the rinky-dink parser was failing to empty the stack, which prevented some metadata from being recorded. (There may be Bonus Bugs hiding in here, because the parser is meant to handle input that I have no intention to produce, because I failed to rein in scope creep.)
I copied code from another project to find elements using xpaths. That code turned out to have never run, and it was slightly broken.
I failed to anticipate the sheer number of ways that Python will pickle "a set full of strings", and had to allow-list a bunch more opcodes.
I realized that I'd allow-listed so many opcodes that I should also look into writing a custom unpickler that only loads trusted globals. The first time, I forgot which global I actually needed to trust.

All of this was ultimately not too bad.

With all of that taken care of, I'm all set to start migrating text into the outline... Except that my eyes hurt and I just really want to lie down and doodle. So I'm going to go do that.

Good night.

Weekly Roundup 2024-03-05

2024-03-05T05:00:00-05:00

Wednesday: I pushed the conversion script in a somewhat more quick-and-dirty direction.
Thursday: I got some stuff done to avoid worrying about recomputations in the conversion script, which is helpful for consolidating logic into one place.
Friday: I made some more progress on the script, and started pondering some of the weird behaviors I've set myself up to think about.
Saturday: I realized that I was approaching said weird behaviors in the wrong way, and found the right way (according to me).
Sunday: I had still more weird design issues to work through, so I did.
Monday: I got probably the last of the big issues with the script, leaving just the little ones.

Next week, I'm going to try and get the script and build process to the point where I'm comfortable migrating content from the flat file into the outline.

Coding 2024-03-04

2024-03-04T05:00:00-05:00

Okay. I was doing other stuff all day, and then in the last hour or so I rushed through implementing most of the stuff I think I need for the conversion script. At this point, I just need to slap a click wrapper on top of all of this, and make sure I wire everything together properly inside.

That shouldn't be too hard, but I want to get some sleep before trying.

Also, this file is a complete mess by now. If I don't give it some editing passes, it'll bother me how messed up it is.

Anyway, I'm going to try and relax and sleep now.

Good night.

Coding 2024-03-03

2024-03-03T05:00:00-05:00

Hm. I mostly messed around today, but I did get some more work done on the conversion script just now. I've got hopefully the right behavior for the title marker at each level of the tree. I've got a bit of work to do in terms of figuring out the runtime checks that I want to try to ensure stylistic consistency. But the big thing I need to work through is how to assemble the text of the different nodes together. Things I think I need:

Some kind of matching argument at the top level
A header helper that does things like temporarily increase the header level
Probably this should all be generators or something?
Chapters and books need to know their own number
I need to figure out where I want end-of-node line breaks to be in all of this. Inserted programmatically, already in the outline, something else?

Thinking about the line breaks... There should be 2+ breaks between everything I think, because I'm not going to want to have multiple scenes in a paragraph, probably. So, I can either emit those as I go, or say "okay, join on double newlines". Stuff to consider for later.

Well, I can figure the rest out later, it's nothing I haven't done before. For now, sleep.

Good night.

Coding 2024-03-02

2024-03-02T05:00:00-05:00

Okay, let's see where I am now.

At the level I'm currently considering, the most complicated logic is "there's an optional subtitle node, and I want to properly handle all of the nodes that are not subtitle." I've got the vague sense that I'm doing something wrong with my design, which is pretty much just in my head right now. Like, I'm currently leaning towards, there's some kind of dependence between the subtitle property and the other attribute. It feels to me like the current draft is wrong, because it's a relatively monolithic setup, and adapting it to the more complicated requirements means I need to duplicate and rework a lot of it.

Let's see if I can divide this up a little...

Hm. I can certainly pull apart different ideas and make the individual units of code smaller, but it's not yet clear if I have the pieces I need to build back up what I want. Thinking about it, the problem I was having before was kind of that I had two monoliths...

Okay, I consolidated a bunch of logic into a compound cached property, and then wrote additional properties to pull out distinct parts of it. This seems like a useful thing to do in this code, for whatever reason; I have some of this in the general Node code as well.

Anyway, with that, plus just knocking out all of the simpler child relationships quickly, I have the tree structure just about handled. Next, I need to get the rest of the data for each node, and then I can work on presentation logic. At that point, it should be ready to move into proper testing.

But for now, I need to sleep.

Good night.

Coding 2024-03-01

2024-03-01T05:00:00-05:00

Okay, I'm in a little better shape compared to yesterday, but I do need to hurry here.

I've improved the basic wrapper class, and I think it's about ready. I've written a mixin class for wrappers around that, which layer semantics around the the helper methods from the basic wrapper. None of those semantics are written yet. I'm aware that some of them are going to need some further tweaks; I haven't really thought about headings yet. Most things are ready to be filled in, but there is one wrinkle I'm trying to figure out how I want to handle:

Chapters can have several different types of children; their first child could be a subtitle, and their children (including the first) can be scenes or summaries. I have an attempt at a helper for handling these constraints, but I don't know that I've hit the right design yet. I'll think about that more later, because right now it's late.

One last thing; here is what I think I need to get done:

Figure out how to handle headings
Figure out the right way to handle children of chapters
Actually write the code to process this stuff out
Migrate draft from the text file to the outline
Update build process

That'll all take some time, and the right thing for me to do right now is sleep.

Good night.

Coding 2024-02-29

2024-02-29T05:00:00-05:00

I was kind of out of it today, but I did make some quick improvements to the conversion script, just about getting cached_property stuff set up so I can feel better about calling some of this code.

Anyway, it's too late and I need to wrap up several minutes ago.

Good night.

Coding 2024-02-28

2024-02-28T05:00:00-05:00

All right. I wasn't focused much on coding today, but I did put together a few things for the conversion script. Basically, I realized, hey, this conversion script is super bespoke and not meant to be reused, so I don't need to take my usual stances against inheritance, so I made a base class for all of my node wrapper stuff, and once I've got that class in a nice place, I can subclass it into all of the wrappers and then focus on pulling out the relevant information.

For now, though, I want to get to bed.

Good night.

Weekly Roundup 2024-02-27

2024-02-27T05:00:00-05:00

Wednesday: I threw together some pretty rough validation code to stop the conversion script from getting pwned by malicious input that there is no obvious way for it to end up on my computer, then started thinking about the actual business logic.
Thursday: I was so gobsmacked by what looks like the way I'm "supposed to" handle one of the requirements I set myself, that I put together a quick hack to see how I get struck down for my hubris.
Friday: I did a little more work with that, and then got distracted by something else I could code up.
Saturday: I prototyped something roughly analagous to μKanren, for... some reason.
Sunday: Traveled out.
Monday: Traveled back.

Next week, I don't know yet what I'll feel like working on; I'll just try to take things as I go.

Diary 2024-02-26

2024-02-26T05:00:00-05:00

Travel back.

Video games.

Nothing to write about.

Oh well.

Good night.

Diary 2024-02-25

2024-02-25T05:00:00-05:00

Traveled today, and then got way into video games. I did do some shell programming this morning, but it's not worth talking about.

It's late and I need to lie down now.

Good night.

Coding 2024-02-24

2024-02-24T05:00:00-05:00

Well, I didn't get anything done on the stuff I was working on just two days ago, so here's what happened instead:

Every once in a while, I read about a programming concept, and just go through implementing it without any clear idea of what I'm going to do with it. This time, it was getting reminded of the existence of μKanren, and trying to put together some Python implementations.

The one that probably works, (though I haven't actually tested all of the primitives, so there could be some really nasty surprises in there) is based around using pyrsistent maps for states, an attrs class that compares by identity for variables, and using iterators for streams. That description basically locks down most of the implementation; I'll also note that I favored callable classes over closures. What remains is the definition of unification, which, well...

The definition of unify and the terms of the language are orthogonal to the presentation of μKanren: both could be changed with limited consequence for the rest of the system.

So, naturally, I figured out how to make that all extremely configurable. Basically, there are some new operators, which register new implementations of unification for the purposes of the goal that they wrap.

I am still not sure what to do with this, if anything, so I'm going to try to switch gears soon. Anyway, I'm starting to feel a little stiff, so I'm going to wrap up now.

Good night.

Coding 2024-02-23

2024-02-23T05:00:00-05:00

Okay, well. I got distracted by something else that I don't feel like discussing right now, but I did make a little progress on the outline-to-draft converter. Specifically:

"Hm, chapter subtitles should probably be a distinct node at the beginning."
And I tested out my weird janky parser. I am aware that it has shortcomings, but I have chosen to only care about them in the event that they become relevant.

Maybe later, I can describe what got me all distracted. But right now, I'm having a bit more trouble than usual typing, so it's time to get to bed.

Good night.

Coding 2024-02-22

2024-02-22T05:00:00-05:00

After yesterday's entry, I remembered that I want to handle Leo directives (by ignoring them) and reStructuredText bibliographic fields (which I'm not overly concerned about getting right, by my standards, but I still need to do stuff). I'm right now trying to figure out the right way to handle bibliographic fields. I could try to hack together a small portion of a reStructuredText parser to winnow them out, or I could attempt to use docutils to just get it parsed out for free. I'm currently not sure which option would be more of a pain, and whether there's some better way.

Let's take a look at the latter option, since it's easier to tell what that entails.

parser = docutils.parsers.rst.Parser()
Store the input in a variable.
document = docutils.utils.new_document("programmatic") (not exactly right, but let's see how this trips me up first)
parser.parse(input, document)

Where the input looks something like

:key: value

Hold on, does something *weird* happen if I have an indented field list inside a code block?

And my hope is that I can extract a mapping of bibliographic fields, and get back the literal text of the rest of it. I do not know if that's a supported usage, so this might be a non-starter. (If it's not, do I have to roundtrip it through xml2rst or something?)

...

AttributeError: 'Values' object has no attribute 'pep_references'

Clearly, I was a fool to trust the builtin documentation.

Okay, maybe the mistake was specifically trusting the default behavior of the settings argument. Okay, that was it, had to get the proper default parser from the frontend module. Now, I have more information than I know what to do with.

After playing around a bit, I see that there are field_list objects, which contain the desired mapping information, several nodes deep. It is possible, then, to pull out those nodes and get the mapping, and somehow convert what remains back to reStructuredText, but...

It's truly impressive to me how this part of the Python ecosystem has managed to convince me that slopping together a rinky-dink little parser would be less trouble than wiring together existing libraries.

...

It is now rather later than I would like, but I have tossed something together that looks plausible. I'll have to try it out later, but it should be at least close. And now, I rest.

Good night.

Coding 2024-02-21

2024-02-21T05:00:00-05:00

Okay, here we are, still dealing with whatever is going on in my head, sinuses, and wherever else. I just threw together some validation code that should prevent my generator script from doing something weird in the face of a malicious pickle. This is, I think, somewhere between hard and impossible to handle in the general case, but in this case, I'm specifically looking for a set of strings, so, like, I think there's less than a dozen opcodes that are valid for that purpose.

The next thing I want to consider is how to handle all of the wrapper-type classes I wrote. Currently, they have attributes that refer to the "child" classes, and keeping that would mean that the nodes would have to be converted eagerly. This seems a little unreasonable when it comes to the case of "convert just part of the tree for spellcheck purposes". In that context, it's sort of justifiable to make the top level lazy, but I don't know which rationale wins at the lower levels:

All use cases will require the whole thing eventually
Doing it all lazily is more consistent
Something about performance characteristics from allocating a bunch of memory up front versus interleaving more kinds of operations later

I'm going to figure "calculating things on the fly means I can add new attributes as properties, instead of as more code in a constructor function". So, on the one hand, I know which way I want to go forward. On the other hand, I've been thinking about this long enough that now it's late and I really want to get to bed.

Good night.

Weekly Roundup 2024-02-20

2024-02-20T05:00:00-05:00

Wednesday: I worked out why and how to generate ninja files. And got a headache.
Thursday: I wrote some helper code for generating ninja files, which is unlikely to help anyone but me. And had a headache.
Friday: I took that helper code and turned used it to write a Python script that generates the currently-used ninja file for my writing project. While I had a headache.
Saturday: I switched gears to remembering how to traverse Leo files. And still had a headache.
Sunday: I figured out how to extract relevant information from the elements in a Leo file. While having a headache.
Monday: I wrote a Node class capable of traversing down, but not up, a Leo file, which should be all I need. Still had the headache.

Next week, I'm going to look into building instances of the higher-level helper classes I wrote before, and into fleshing out their behavior so I can actually be useful. And dabble with a few other projects. And take some NSAIDs, sheesh.

Coding 2024-02-19

2024-02-19T05:00:00-05:00

All right. I think I've covered the adaptation layer over lxml and Leo, so now what's left is taking my draft of the higher-level concepts, and fix them up so they work with the wrapper code I wrote.

I figure I'll describe what I wrote today in a bit more detail:

The Node class now just has an attribute I called t_ref, which corresponds to the t and tx attributes.
Now there are a bunch of properties, which do things like:
Get the xml element of type v or t, depending on what's needed;
Get the title
Get the text
Get the tags
Get the children

I've got nothing else I want to think about on this right now, so I'm going to wrap up this and work on other things for a bit.

Good night.

Coding 2024-02-18

2024-02-18T05:00:00-05:00

Okay, I got inspired to look more into generating stuff from Leo files. I think I mentioned I wanted to key some behavior off of tags, so here's what I found looking at the file.

Node tags are stored in t elements, in an attribute called __node_tags. This attribute contains a hex-encoded string; when decoded to bytes and unpickled, the result is a set containing the tags as strings.

So, assuming I remember the relationship between v and t elements correctly, v tags lay out the tree structure, and point to t elements, which contain the node text and other metadata. Of the v tags that point to a given t element, just one contains the vh element that gives the node title. I could be wrong about this in hopefully subtle ways.

However, for the purposes of this script, I'm going to generally ignore the node titles, except to find the root of the outline. So, I want to say "for each v that is a direct child of vnodes, find all v that are descendants of vnodes, such that their t attributes match, and they contain a vh element that contains the string "Draft"". Looks like the child information follows the vh element, so I will need to perform that search in all cases, in case of something non-standard going on with the tree/graph layout.

In terms of actually printing this stuff out, I should avoid loops, which my whole "tag what kind of node it is" idea should get me for free.

Anyway, I've just about run out of juice for today, so I'm going to wrap up.

Good night.

Coding 2024-02-17

2024-02-17T05:00:00-05:00

Still dealing with stuff today. I'd like to work on the conversion code, and one part of that is reminding myself how I wrote some similar code. Let's see what we've got here...

Parse the file
Find "vnodes" and "tnodes"
For all "v" in "vnodes", I may need to use a helper function to get "the" copy of a "v", the one with the "vh"
The structure I'm going with wants the top-level "v" with "vh.text" of "Draft".

To get much further than this, I'm going to need to put together pseudocode for how this should all fit together, fill in the outline enough to be able to execute it, and work through it all manually-ish.

I'll try to get on that tomorrow; I'm once again really sleepy. I'm not totally sure I stopped being sleepy today. Therefore...

Good night.

Coding 2024-02-16

2024-02-16T05:00:00-05:00

Hooray, I took the weird code I wrote yesterday, and created a version that will regenerate itself if I edit the Python script I wrote.

At this point, there are a few directions I can take this. If I can make a better "python" rule, then I can split up the script into smaller files and have them all properly tracked. That seems pretty straightforward to specify and put together. (Although if I push it too far, then it kind of makes a case to try switching to tup...)

The bigger thing to look into is figuring out how I want the generation scripts for the actual artifacts to work. I don't have the focus to think about that right now, so I should get ready for bed instead.

Good night.

Coding 2024-02-15

2024-02-15T05:00:00-05:00

Okay, I can't really do too much today, but let's see what I can work out for later.

I want to write a Python script to generate my build.ninja file. In service of this, I have installed the "ninja_syntax" package. The key thing for me to use from that package is the Writer class, which requires an object with a write(str) method, and a close() method. A text file, basically. I have a set of rules and build statements to generate. My initial thought is that I should have some kind of helper method for the build statements, so I can just call some_object.pandoc("story.rst", output="build/story.html").

The idea of wrapping these disparate objects around each other like a matryoshka doll is making my teeth itch a little. Kind of makes me want to replicate the Compendium creation code from MOTR, but that's probably extreme overkill, and yet...

... I have just discovered that the interface to ninja_syntax has changed somewhat over the versions, and I assume I should probably just vendor a version that matches my installed version of Ninja.

...

Okay, after vendoring the file and doing the thing, I should be all set to start writing code to generate a build.ninja file. Sadly, it is late and I am feeling bad in addition to feeling tired, so I'm going to go lie down.

Good night.

Coding 2024-02-14

2024-02-14T05:00:00-05:00

Me several months ago: "Why does the Ninja documentation warn you against hand-writing the files? This is really easy and pleasant."

Me now: "The prospect of writing rules for anything remotely tricky makes my skin crawl. ... Oooh..."

So, yeah, I'm at a bit of a juncture here. Either I can write code to generate some gnarly nonsense for Ninja to work with, or I can try out something else, like, see if tup acts any nicer when I have ideas like "well, I might want to change the code to translate the draft between different formats, and that should force the build to redo, so I guess I've got to write files like—"

rule python
    command = python $args > $out

build build/preface.rst: python build.py draft.leo
    args = build.py --section preface draft.leo

Or however Ninja works, I didn't actually try that out, in part because build.py isn't fleshed out enough to do that yet.

In any case, I think I basically understand how to try and put stuff together for generating Ninja code, which makes it somewhat unfortunate that I feel kind of bad right now actually, and just want to go lie down.

Good night.

Weekly Roundup 2024-02-13

2024-02-13T05:00:00-05:00

Wednesday: I tried to get the logic right for handling "absolute convergence requires that only countably many values are non-zero".
Thursday: I wrote really weird code.
Friday: I explained why I wrote the really weird code, and what I think makes it weird.
Saturday: I got sidetracked away from Dafny to work on build pipelines for a novel draft.
Sunday: I tried to figure out the minimum I need from these pipelines, so I can get back to writing.
Monday: I went over what I'm trying to do there again, and put together the beginnings of a basic idea of how this all should work.

Next week, I'll keep working on something, that's for sure.

Coding 2024-02-12

2024-02-12T05:00:00-05:00

First pass at how to represent all of this:

There is a draft, made up of multiple documents.
The document type can be a union if need be, but I'd like to try to come up with a protocol.
Specific types are... let's say FictionOutline and ReferenceWork. (The latter is needed for a factual description of the neopronoun usage. That... shouldn't have character arcs.)
For now, I want to structure the ReferenceWork pretty loosely; the current draft just has thirty-something lines of text that should appear in the preface, so I'd like to be able to just slap those into a node and call it done. Hm. Not sure how I want to set up the titles... This all seemed obvious in my head, but somehow I ended up tying my mental model into knots.

Like, if I say "the pre-processing should strip out Leo's @ directives", then I can include metadata at the topmost level and title it like that.

All right, thinking about this more... It might make sense to just work with undifferentiated node types to start with, and then work on narrowing things down as I find unique behavior/validation logic. Although I can kind of anticipate some things. Like "I want to define a bunch of metadata for each chapter, such that there is a set of key names for which any given key is either given a value by every chapter or none of them".

So, I can try working with:

Draft
ReferenceWork
FictionOutline
FictionBook
FictionChapter
Scene
Summary

The next thing to figure out is where I put all of this stuff. On reflection, it probably shouldn't be in the tasks file, since I'm going to want to call it from the ninja build, so I guess I need to set up a virtual environment and requirements file for this stuff. I'll get on most of that later. Right now, I once again need to get to bed.

Good night.

Coding 2024-02-11

2024-02-11T05:00:00-05:00

Okay, writing this early in case my laptop has another little accident like it did about half an hour ago. I've currently shifted gears to considering how I might convert my disorganized draft file into a more comprehensible outline file.

Here is the hierarchy as I see it at a glance:

Top-level: title
Second-level: preface; the preface contains explanatory notes about the word choices in the book(s); it therefore is needed to understand/contextualize the story proper, but it does not fit into the structure of the rest of the outline; spell-checking it will require a different dictionary from the rest of the book; there are some parts of the "preface" that are more properly "notes", and should be transcribed into a separate branch of the .leo file's tree structure.
Second-level: main story; the actual contents here are a muddle of notes-to-self, outline fragments, actual text meant to show up in the draft, and who knows what else; of interest from the notes is that I call out how a specific event should be followed by a chapter break.
Lower-level: chapters; not currently planned out except in the broadest contours; I'm aware that whether to have chapters is a choice, but the choice of chapter break I have in there is really amusing to me, so it's a choice I'm making in the affirmative; I'll reconsider if I end up with "Chapter 2: Everything Else"; the other thing I need to ponder some is the question of chapter titles, subtitles, quotations, etc.
Lower-level: "scenes" and "summary"; Rather than my draft, this is based on the writing advice I've been reading; The idea here is that I would like to reason about smaller parts of the narrative than a single chapter, so I should call them out here.
Other levels: I am aware that I could come up with groupings of chapters (I probably will end up needing "book" to be honest, because the draft ends on a sequel hook), but I don't want to get ahead of myself; similarly, maybe there are divisions that I'd like between chapter and scene, or within scene, but my gut feeling is that I should try working with book-chapter-scene, and see what that gets me; (the obvious things that occur to me are "paragraph" and "sentence", which I hope illustrate why I think it may be best to hold off on dividing things up that finely).

If we assume that I'm going with book-chapter-scene, then I've got some ideas for what I need to get things right.

Books must have a title, and this title must end up in the generated documents.
Chapters may have a title etc, which should end up in the generated documents.
I would like chapters to be consistent about what such information they have associated with them.
Chapters are also numbered.
It is likely but not certain that I would like to associate different kinds of metadata with scenes, such as which arcs they relate to.
Some of this metadata is best expressed through references to scenes in other parts of the tree.

Summing things up:

Every division between book and chapter, inclusive, represents a visible distinction in the final output, including, for example, specific formatting to call out the boundary.
These nodes are to be named with a word or abbreviation indicating their role (for verification purposes), and a short summary of their contents, for aid in navigating the tree.
The contents of these nodes are to consist of some lines of metadata, followed by a more detailed summary of the child nodes.
The contents of a node should be fully accounted for by its direct children.
For evaluating the outline, I would like the ability to mark specific nodes as "included literally" or something; such nodes should not be allowed to have child nodes.

I wish to process all of this into a few different forms:

preface only
story without preface
story with preface (which is then processed into other forms)
absolutely everything (not sure the exact way this should present everything, so I'd like to hold off for now)

To accomplish this, I'm pretty sure I need:

Use of node tags for repeated scalar or mostly-scalar metadata.
Use of inline metadata for key-value metadata, unless there is some alternative.

I think this is all solid as an idea, but I need to take some time to sketch it out on paper for a bit before I can implement it. And none of that is happening now, because it's late again.

Good night.

Coding 2024-02-10

2024-02-10T05:00:00-05:00

Okay, this is interesting. I don't have the proper context for putting things together in terms of absolute convergence, but when I tried drafting the required predicates, a few things jumped out at me:

The use of "counting" values makes the structure of the predicates simpler.
This simplification means that I need different predicates supporting it.

I'll need some time to collect my thoughts on this. So, for at least a little bit, I'm going to put this project down and focus on some other things.

In particular, I've got a novel I've been writing, and the current draft document is unwieldy to navigate. I've been reading about writing advice, and trying to figure out how to change my process to address the issues I'm having.

Currently, I have a reStructuredText file that contains a combination of "actual draft text", [text in brackets] that indicates summarized ideas of what could go in, and reStructuredText comments that indicate structure that doesn't have either of the former two backing it up. I have a tasks.py file that defines tasks to check the spelling of the file; this is necessary because I need to allow some words, and forbid others. (Weirdly enough, when I was trying to forbid particular words, hunspell wouldn't acknowledge those words, so I needed an extra step to find them with a regular expression.) And another to convert the file to html and docx, as handled by ninja, via a hand-written file.

What I'm considering trying to do is to convert the draft to an outline in Leo, and write code to process the .leo file into an rst file that can then feed through the other steps. What I need to figure out before embarking on trying to do all that is, what structure do I want to put the outline in? What would be helpful for me?

Here are my quick thoughts, untested ideas:

Have the outline correspond to the hierarchical organization of the draft.
Figure out how I might attempt to add in some helpful metadata, but don't worry too much if I can't figure out how to make that practical.

So, going forward, not now, but soon, my plan would be to transcribe the draft and some scraps of exposition into a .leo file, with one section for the outline, and another for notes. Sort of like a very un-flashy Scrivener file, since I don't feel like resuscitating an old laptop and trying to get Scrivener working again. There are still some questions to answer here about the details of the structure, and I'll try to figure them out later. For now, I've pushed things way too late.

Good night.

Coding 2024-02-09

2024-02-09T05:00:00-05:00

Here we go, I've verified that my weird Dafny code appears to work, so I'm ready to talk about it.

The goal: Functions that return other functions which meaningfully depend on their outputs.

The problem: Dafny appears not to support creating closures. I tried the workaround of "make an object", and that ran into "can't allocate in functions" (which, from my basic experience with Rust, I guess makes sense (please note that Rust functions and Dafny functions are pretty different things)), but also "I tried returning a bound function from an instance, and that didn't seem to work, oh well".

The messed-up workaround: While Dafny, if I'm reading the grammar documentation correctly, doesn't have a way to declare a function inside a function, it does have the ability to use ensures clauses to describe what properties a function capable of doing such a thing would have, and then just, don't write a body for the function, and it will successfully reason about whatever currying-based nonsense you ask it to.

I suppose another possible approach would be to write a predicate describing the desired relationships, then using the predicate with an assignment statement to create a fake closure through Prolog Nonsense. I think that approach would be more flexible in some key ways, but I don't think I need them, so I'll keep on with what I'm doing now. And if I do end up needing the flexibility, I can write the predicate, replace the ensures clause with it, and move on.

...

All right, I've put together a few more of these helper functions. At this point, I'm just about ready to redo the absolute convergence predicates, knowing what I now know about how I want to set things up. I could try getting on with that now, or...

Or I could say "You know what, that's enough for now, I want a break from screens and to get ready for bed." Probably the smart play.

Good night.

Coding 2024-02-08

2024-02-08T05:00:00-05:00

Okay, so.

It turns out one of the things I wanted to do last time isn't straightforward to accomplish in Dafny, but I may have found a way to do it. An extremely cursed, messed-up-seeming way.

I don't feel like describing what I was trying to do, or verifying that my attempt at a fix actually works. I'll do that later.

For now, I'm going to read and wind down.

Good night.

Coding 2024-02-07

2024-02-07T05:00:00-05:00

So, last time, I—

Ahem

mentioned that working with absolute convergence requires coming up with a means of counting the non-zero elements of an infinite set.

So, to try to formalize this before I trip all over myself in the code...

Given a set S and a partial function σ from S to ℝ. We want a pair of functions between S and ℕ, such that:

For a given s in S, if σ(s) is non-zero, s is mapped to an integer n, and n is mapped back to s.
σ(s) is defined for all s in S; extend σ by giving it a zero value for all input values not in S, thereby producing a total function.
For a given n in ℕ, if n is mapped to an s in S such that σ(s) is non-zero, then s is mapped back to n.

The computational artifact of interest is the mapping from ℕ to S. Let us assume it can be a partial function. We can combine such a mapping with σ to obtain a total function from ℕ to ℝ, which, if the sum is absolutely convergent, it is identical to summing over S.

We can avoid having to define an explicit inverse function, by using existential quantifiers, I think.

σ(s) is non-zero implies that there exists an n, and this n must be unique, which is equivalent to "for all n' that correspond to s, n == n'".

I'm going to need to think carefully about whether the first and third bullet points are equivalent, because I tried to relax the requirements from "inverse functions", and it's muddling my thinking. Right now I'm leaning towards "the previous paragraph is all I need", but, we'll see after I've had some rest.

(Since I've thought about this hard enough that now I just really really want to switch gears and let it all percolate for a bit.)

Good night.

Weekly Roundup 2024-02-06

2024-02-06T05:00:00-05:00

Wednesday: I wrote up a list of properties a binary relation can have, that seemed either useful or easy to describe.
Thursday: I learned the basics of Dafny project files, which seem really useful for anything involving more than one file, and anything involving particular tweaks to the verification process.
Friday: I tried to prove some bedrock theorems in Dafny, and had a bad time with that, but I did learn a little about the module system.
Saturday: My day was too busy to do much that interested me.
Sunday: I pulled Demiurgent Business out of mothballs to publish the jam and jam-adjacent versions, which aren't really fleshed-out, or tested, but they are short, at least.
Monday: I figured out how to prove that bedrock stuff from earlier.

Next week, maybe I'll keep on with this, maybe I'll wind up glomming onto one of the other half-dozen-plus projects I have lying around.

Coding 2024-02-05

2024-02-05T05:00:00-05:00

All right. I don't have time to explain it, but, by following some suggestions online, I managed to prove that summing from a series with indices drawn from a set, the order that the indices are chosen doesn't matter.

Only abstractly, and not with IEEE floats as the output, of course.

Anyway, with that, and a few other things I wrote, I'm most of the way towards being ready to work on the absolute convergence stuff again.

Doesn't the focus on countable sets in the context of absolute convergence require some form of bijective mapping to the naturals, thereby rendering the whole detour to avoid imposing an ordering on the sets of indices pointless?

At least I got some experience wrangling Dafny.

Anyway, it's late and I'm tired.

Good night.

Demiurgent Business 2024-02-04

2024-02-04T05:00:00-05:00

This weekend is probably going to be a bit of a wash, so I decided to put out some stuff I put on my tumblr a few months ago. See, during my NaNo (which failed for various reasons that I want to think about at some point...), there was a 200-word RPG jam, and I ended up taking some time to strip Demiurgent Business to the barest essentials. I ended up with two versions:

Jam-version-1

Demiurgent Business

For 3+ players

Each player picks 1 [Supernatural] trait and 1 [Character] trait, then names and describes the resulting character.

Example [Supernatural] traits:

Time-traveling
Android
Psychic
Alien

Example [Character] traits:

Honor student
Athlete
Slacker
Tech whiz

The group designs the Demiurge: agree on two [Character] traits, then name and describe the resulting character.

One player starts as the Demiurge; pass the role clockwise when a new scene begins.

When a scene begins, set Gnosis to 0. The Demiurge describes the scene's goal. The other players help the Demiurge fulfill that goal.

The Demiurge describes obstacles that arise. The other players describe how their characters resolve the obstacle. For every [Character] trait involved in the resolution, decrement Gnosis to a minimum of 0. Then, for every [Supernatural] trait involved, increment Gnosis. Whenever Gnosis increases to 3, the Demiurge may make 1 mundane change to the setting. Whenever Gnosis increases to 6, the Demiurge may make 1 mundane or miraculous change to the setting. When Gnosis increases to 9, the Demiurge achieves Apotheosis; the Demiurge's current player wins. Otherwise, the scene ends when the other players determine that the goal is completed, or impossible.

Too-long-to-be-jam-version-2

Demiurgent Business

For 3+ players

Each player picks 1 [Supernatural] trait and 1 [Character] trait, then names and describes the resulting character.

Example [Supernatural] traits:

Time-traveling
Android
Psychic
Alien

Example [Character] traits:

Honor student
Athlete
Slacker
Tech whiz

The group designs the Demiurge: agree on two [Character] traits, then name and describe the resulting character.

One player starts as the Demiurge; when a new scene begins, the player who relied most on their [Supernatural] trait in the previous scene becomes the Demiurge.

When a scene begins, set Gnosis to 0. The Demiurge describes the scene's goal. The other players help the Demiurge fulfill that goal.

Commentary

First off, I haven't tested either version of this, so it probably has issues. In particular, there's the issues around the differences between the two versions. I made the changes because I wanted to try to make play more fast-paced and competitive, but I'm worried it could lead to a kingmaker dynamic.

Going forward, I want to try to come up with tables to roll on for [Supernatural] and [Character] traits, as well as the ability to mad-libs together a scene's goal, so that sometimes the players have to figure out what sub-goals and obstacles there may be to accomplish "Film a Website".

And once that's done, there's the possibility of putting together alternative tables for [Character] traits and goals, allowing for supplements that are somewhat analogous to Fiasco playbooks, thus allowing for scenarios like "The Demiurge is Beret Guy from xkcd, and the player characters are employees at his strange start-up."

Anyway, I'll keep that all in mind. For now, I'm actually writing this early and basically banking it against tonight. Maybe I'll add something after we travel, maybe not. We'll see.

Nah, I'm good.

Good night.

Diary 2024-02-03

2024-02-03T05:00:00-05:00

I had a whole bunch of stuff that I had or wanted to take care of today, and that ultimately translated into "No time to consider stuff for Dafny."

I'm traveling tomorrow and the day after, and my plan with that is to either get something done with Dafny (either proving some basic facts, or fixing up the files I wrote before I understood what project files were capable of), or to write up some stuff from a few months ago that I may as well at some point, hopefully soon.

Right now, though, I want to close stuff down and get to bed.

Good night.

Coding 2024-02-02

2024-02-02T05:00:00-05:00

Instead of rewriting my following-along code, I wrote an IsTotalOrder predicate in terms of named properties, and now I'm thinking about the finite sum stuff. What I'd like to figure out is if there's some way to establish the equivalence between the explicitly ordered non-ghost version, and the unordered ghost version.

Going to speculate on ways to do that.

Suppose I do induction that "builds up" stuff. Given a set s and an order r, I want to build things up something like:

If s == {}, then both versions sum to 0.0.
Suppose both versions have the same sum for some s. Now, given an element k (not in s?), I want to try establishing that s + {k} increases both results by summand(k).

Hm, I've got some time to try this out. I also need to get a handle on the import system in Dafny, but I want to work that out at some point anyway...

"By default (in the absence of any export set declarations) all the names declared in a module are available outside the module using the import mechanism."

Neat, thank you for that.

"Error: module Relation does not exist"

No thank you for that. What am I doing wrong here... Tinker, fiddle...

"Error: Import declaration uses same name as a module in the same scope: Relation"

Huh. That implies that it doesn't really work the way that I think it does... Let's try what it looks like that should allow.

Huh, yeah, module names are just globally available, and import does rebinding? That... freaks me out a little bit.

Anyway, I got the code reuse wrangled, copied over the code, and now I'm working on constructing a lemma to prove that addition of a finite collection of numbers works the way everyone knows it does. Dafny does not yet know it, at least not in a way germane to the way I phrased the lemma.

Unfortunately, it's now late, and I'll have to wait until sometime tomorrow to get this worked out, if not later.

Bed.

Good night.

Coding 2024-02-01

2024-02-01T05:00:00-05:00

I was kind of out of it today, but I managed to get all of the baseline definitions I was thinking about in. At this point, I just need to define the total order stuff, and move along.

The more important thing is, I now get the (very very very) basics of Dafny projects. At this point, it makes sense to try to rewrite my existing "following-along" code to be consistent with the style guide, and to try to handle verifying it with as few project files as possible.

Well, I'll get to that later, and get to bed now.

Good night.

Coding 2024-01-31

2024-01-31T05:00:00-05:00

I've got other stuff I want to put up entries about this week, but today was a lot, so instead I'm going to do some planning for the module separation I want to do.

I'm right now focusing on the total ordering relation, and for the fun(?) of it, I'd like to define that predicate in the context of other predicates applying to binary relations. Let's see which properties may be of interest:

Reflexive and irreflexive
Antisymmetric, asymmetric, symmetric
Connected and strongly connected
Dense
Transitive and antitransitive
Quasitransitive? Like, I can define it, but I'm not sure offhand what I'd do with it.
Serial? Like, sure.
Euclidean?

These properties should be a good baseline for putting things together, and then building more elaborate properties on top of them.

I'll try to get on that in the next few days. For now...

Good night.

Weekly Roundup 2024-01-30

2024-01-30T05:00:00-05:00

Wednesday: I had some troubles with Dafny, so I set my sights a little lower.
Thursday: Screen headache.
Friday: I started trying to take rigorous definitions of limits, and convert them to Dafny code.
Saturday: I came up with a game plan for (part of) the infinite limit representation.
Sunday: I ran into bizarre issues with Dafny that sound like bugs, or at least missing features, but I lack the context to understand the stuff about how it's "supposed to work", so eh.
Monday: I got past the issues, made everything verify, then looked at the file and thought "that's too big; I should learn how Dafny projects work so I can break this into at least four files".

Next week, I'll probably do that; I also have a few other things that I've started messing with recently, and a few things that I've been messing with on and off.

Coding 2024-01-29

2024-01-29T05:00:00-05:00

I did a bunch of admin work stuff today, and then played video games, so I haven't done a ton of work with the Dafny code. Instead of trying to rush into anything, I'm going to plan a little for later:

IsTotalOrder seems like a generally useful predicate.
ThereIsAMinimum seems like a special-purpose lemma, which should get bundled up as a private member of the module that provides...
SumOver, which I would like to figure out some way to provide an equivalent ghost version of that doesn't need an ordering. (The trick is proving that the two versions are equivalent.)
LessEqual is a basic helper predicate that bundles with some of the predicates below, but I wouldn't mind getting rid of it if I work out the proof I mention above.
I've got a bunch of helper things that I don't know how generally useful they are: HasMinimum, and Minumum are needed for how I'm trying to define absolute convergence, but I don't know if anything else has reason to refer to them.
IRHelper is only needed because InclusiveRange was having troubles; wait, can I define nested functions in Dafny, because that would be handy here. Either way, these together represent a single coherent concept that seems generally useful. (Part of that usefulness is that I couldn't be bothered to figure out the right number theory proof for "there are a finite number of integers between any two integers, and if your response to that is "that sounds like something that the set comprehension heuristics should give you for free", the answer to that is "somehow, not the way I was using them".)
SetUntil is a helper for absolute convergence stuff, but it's probably needed as part of proving absolute convergence in any specific case.
NonNegativeAtValues is a predicate needed for absolute convergence stuff; I'm not sure offhand that there's any use exposing it.
Then, I've finally got three predicates with really unwieldy names; these build up to the idea of absolute convergence, and until I try actually establishing absolute convergence of basic series, I won't know how generally useful they are to refer to.

So, that works out to four-ish modules I want to break this stuff into:

total order and anything related to that somehow
finite sums
inclusive ranges
limits of absolutely convergent sums

And, with that, I'm going to call things done for now, and get ready for bed.

Good night.

Coding 2024-01-28

2024-01-28T05:00:00-05:00

Okay, after a lot of struggle (something seems to be wrong with the whole "heuristics" thing that set comprehensions have, and some of my efforts to fix it landed me on StackOverflow reading about "triggers", and completely confused because this was happening in set comprehensions rather than a forall, so I couldn't figure out how to translate the stuff about triggers into this context), I've got the functions from last night implemented and the code somewhat cleaned up.

Now, for the rest of the cleanup, I need to figure out how to set up modules and imports so I can slim down these files and come up with better names.

For the moment, I need to wind down, now that I've accomplished a bunch of stuff, at great psychic cost.

Good night.

Coding 2024-01-27

2024-01-27T05:00:00-05:00

Well, shortly after last entry, I remembered that forall exists, which will probably be helpful to writing this stuff up in Dafny.

The challenge I have now is that I'm not sure what to call some of the predicates I'll be writing.

The major thing I want to work towards is a predicate for "this value is the limit of this increasing series as the index increases to infinity".

This is equivalent to "for every positive real number, there is a finite number of terms that add up to between that number and the limit, but no finite number of terms that adds up to strictly more than the limit".

So, we want to say that, given a positive real number and everything else, we can find a minimum value above which the series is bounded between the limit and the limit minus the epsilon.

And finally, we want to say whether, given a specific minimum and everything else, that set of values describes a valid example of the definition of a limit.

From this, we can then build out predicates for more interesting sets and summands.

I'm not going to try and implement this right now, but I'd at least like to try to come up with some names.

InfiniteLimitOfIncreasingSeries
IndexBeyondWhichIncreasingSeriesIsBounded
BoundsOfIncreasingSeriesBeyondIndex

Or something like that.

I've got nothing more in me tonight, so I'm going to try to wrap up soon.

Good night.

Coding 2024-01-26

2024-01-26T05:00:00-05:00

Well, I feel awful, but I also feel stubborn. Let's see what I can make of infinite sums that converge absolutely.

First off, this is all ghost functions unless I get seriously inspired.

Next off, a lot of the math for this (somewhat reasonably) involves infinite limits. But those only make sense when the set being summed over is a subset of the integers. The dream is that there is a general notion of convergence, and a lemma or something that shows that constructs involving limits satisfy it.

Before anything else, let's consider what's needed to express one of these limits, because that's probably going to be tricky.

For these infinite limits, let's start by only caring about positive infinity. And assume that the inputs to the primitive version we start with are non-negative. These assumptions will probably have to be unbundled later, but I need to prototype something.

With all of this, if we say that some infinite sum over a subset of the integers with a minimum value (another assumption), then the limit of that sum as more integers are included is...

Ugh...

Given an epsilon, we can determine an N, such that the sum of the subset up to any number above N will always be within epsilon of the limit.

This sounds like a negative statement, and I'm not sure where establishing a negative in Dafny stands in terms of pain level, ranging from "trivial" through "confusing" to "impossible". Perhaps some form of inductive proof... Like, if I can establish that, for a given function, having the inequality hold for some number M means I can establish that it also holds for M+1...

So, that provides a way forward to answering the question; now I just need to figure out how to ask it.

So, we need a subset of the integers.
We need a function from the integers to the reals.
We need the values of the function to be non-negative for every integer in the set.

Now, function plus infinite set does not have a general algorithm. So, I think this needs to be some form of lemma or predicate or something.

Like, a predicate that requires some inductive stuff.

ghost predicate Lim(nums: iset<int>, summand: (int) -> real, limit: real, epsilon: real, N: int)

Or something. The idea is that it would require the nums set to have a lowest element, that every element k in nums has a non-negative value for summand(k), and that if we take the finite subset of nums less than N, then the finite sum of summand applied to the elements of that subset must be less than limit, and that sum plus epsilon must be greater than limit, and the same predicate must hold except with N+1 instead of N.

I'm not sure if I'm able, at my current level of abilities or in general, to express that idea and make use of it, but it seems like a promising avenue.

And, I am wiped out for tonight and ready to take a break.

Good night.

Diary 2024-01-25

2024-01-25T05:00:00-05:00

I messed myself up today staring at screens, so now I need to stop looking at my laptop, and I'm not currently doing a great job of that.

It would be nice if I thought of something else for this entry, but nope.

I need to go grab a book or something.

Good night.

Coding 2024-01-24

2024-01-24T05:00:00-05:00

Guys, it is really hard to write this stuff incrementally when I assume that "not filling in the branches of an if statement" will result in some kind of verification error, and instead it just makes my laptop fans try to launch it through the ceiling.

I've changed things up a bit:

a ghost function to specify behavior; I think it assumes that Dafny's real type is associative and commutative under addition
a ghost predicate for stating that a dyadic function acts as a total ordering
a lemma stating that any finite set has a minimum according to a given total ordering
a summation method that requires a total ordering to fix the order of execution

I am doing something wrong with the last bit, but I can't tell what.

...

A-ha! I neglected to invoke the lemma in the method, which caused Dafny to go into an infinite loop because I don't know!

A few more tweaks later, and now I just need to determine why Dafny appears to think that 0.0 might not be equal to 0.0.

...

I think I understand the gap between what I'm trying to get Dafny to accept and what it has available to work with, but I don't want to deal with it...

So I didn't. I stripped out the ghost function and made the "regular" function be the entire implementation. I'll try not to care about that too much.

Once I got the code all verifying, I figured out which asserts I could get rid of, which turns out to be risky because the ones I couldn't get rid of put it into an infinite loop again.

So, the code is, if not clean, at least sort of short. Maybe I'll clean it up later. Maybe I'll move on to other bits of code. Maybe I'll decide to do something else entirely.

I'll work that all out later, and for now get to bed, because sleepy.

Good night.

Weekly Roundup 2024-01-23

2024-01-23T05:00:00-05:00

Wednesday: I started looking into summation, and thinking about how to handle it in Dafny.
Thursday: I narrowed in on the thing that I absolutely need to figure out how to do in Dafny, in order to use it for verifying this math.
Friday: I took another look at the chapter in the textbook, and things ended up going a little out of control.
Saturday: I discovered that interacting with sets in Dafny is... strange.
Sunday: I found a few things that I had been doing wrong with Dafny, and then ended up doing a few other things wrong.
Monday: Got sick.

Next week, I'm not sure what I feel like doing. I did discover that whatever I'm doing wrong with Dafny is orphaning z3 processes that are squatting on one core each-ish, so that's... not ideal. As far as Dafny goes, I want to try and figure out at which point the hang is coming into play. As far as other things go... I'm not sure, we'll see.

Diary 2024-01-22

2024-01-22T05:00:00-05:00

Well, I'd been thinking of trying to work on Concrete Mathematics and Dafny today, but I relaxed most of the day, and tonight, my body came up with other, much less pleasant plans.

Now it's much later than I meant to be up, and there's no way I can get anything else done. I'm going to hope that things work out a little better tomorrow, but let's not force anything.

Good night.

Coding 2024-01-21

2024-01-21T05:00:00-05:00

Okay, still getting to grips with Dafny. It's not a fast process when I'm just doing a little a day, I guess. In any case, I now understand something that had me a little confused. The meaning of the keywords kind of changed around between Dafny 3 and 4, and a lot of the stuff I'm reading happens to be for 3, but I'm using 4 because I wanted to make some use of the standard library.

In particular, what 3 expresses via function method is just function in 4, while what 3 calls function is ghost function in 4.

In light of this, I've slightly updated my existing files to make sure that they're actually expressing what they're supposed to; basically just making sure that the recurrent forms are ghost functions.

I also ran across some documentation about telling Dafny to generate counterexamples; I'll have to try that out later.

Aside from that, I tried to put together some flexibly generic code for summing over sets, using a short guide I found about iterating over a collection, and I'm not sure what I'm doing wrong, but my attempts to follow the directions seem to be putting Dafny in another infinite loop, so that'll be some kind of pain to troubleshoot.

(At least the infinite version should be "easier", since I think that has to be ghost.)

Also, thinking about this, I'm going to have to remember how to rigorously prove convergence, and that could be... a thing.

I may end up needing to take a break from this and focus on somewhat less intimidating pursuits. For the moment, I should get ready for bed.

Good night.

Coding 2024-01-20

2024-01-20T05:00:00-05:00

"Hey, what if I handle finite sums as a simple function, since those don't need to worry about convergence tests?"

"Hey, why don't I make the baseline implementation use finite sets, for flexibility?"

"... I don't understand how to iterate over a set in Dafny, possibly because I'm now too tired to read the documentation properly."

"After writing that last line, I figured out where to look in the documentation, and now I kind of wish I hadn't?"

Okay, so, if I want to do all of this in a function, I need to extract an element via a bizarre sequence of glyphs, then set difference it out and recurse? At least I don't need to worry myself about tail recursion, since I'm not planning to execute this. I might do it anyway, just because it's not like I can make this any harder to grasp.

Anyway, I'm going to consider it a win that I worked out that much, and try to get ready for bed.

Good night.

Coding 2024-01-19

2024-01-19T05:00:00-05:00

Okay, let's see.

Delimited summation notation has a lower and upper bound on an integer, and a function from an integer to a real.

Generalized summation notation has an iset (should there be something separate for specifically a set?) of some type T, and a function from T to real.

It may be pleasant to include a form that combines bounds with predicates, but I'm not sure how many "nice-to-haves" there are to lay out. Like, a single integer index, with lower and upper bounds, plus a predicate, seems sensible.

Then I read a little further, and remember some things that make it not clear to me how things should proceed. Like, working in Python-style pseudocode because I don't want to look stuff up, suppose we have Sigma(a, p), and I want to change it to Sigma(lambda k: a(k + 1), lambda k: p(k + 1))... That looks vaguely reasonable, although for the actual code I'll probably want something like a shift operator for this specific case?

The sense I'm getting is that this should be a "sandwich" type of thing, where the "boundaries" prefer simple delimited summation to delimited summation with a predicate, to summation with a predicate or iset or something.

Now, the chapter is discussing Iverson brackets, which I want to represent as a function from bool to a function from real to real.

A tape player?

Moving right along, I suppose the second function there would ideally have a default value of 1.0.

Although, maybe it's not a function, because we see also the possibility of "summation with no explicit predicates, but a bunch of Iverson brackets in the summand". How all of that gets represented depends on how much Dafny can "look inside" functions to pull stuff out. I'll have to experiment with that later. For now, let's assume it can, and try to contrive a situation where it's obvious if that's wrong.

Anyway, I got distracted and it's late. Better get to bed.

Good night.

Coding 2024-01-18

2024-01-18T05:00:00-05:00

All right. Skimmed to the end of the Sums chapter, and now it's time to look at it again, more carefully.

I'm currently sort of vacillating between "in Dafny, just go all in on doing arithmetic with real values" (and hope there aren't precision issues when it comes to doing purely verification), and "okay, but the chapter mentioned complex numbers, so I should really try to define traits to support algebras" (I have no experience writing traits in Dafny).

The reasonable course of action is to see how far I can get with "a data type that includes integral bounds, optional predicates, and a function from integers to reals". It's been really easy for me to get caught up in thinking about stuff like "Oh, this operator is basically a predicate followed by and in Python", and lose sight of the fact that it's just as important, if not moreso, to have tools for manipulating these forms, and not just evaluating them.

It's way too late to mess around with code right now, but here's the main question I think I'd like to address:

Given some format for expressing an infinite sum, is it possible to express the predicate "this sum converges absolutely"? Furthermore, can I properly express the implications of that for reasoning? If I can't figure out how to do that, then it's probably not worth trying to manipulate the sums programmatically.

We'll see where I get with that in a few days, hopefully. For now, it's later than I meant to be writing this.

Good night.

Coding 2024-01-17

2024-01-17T05:00:00-05:00

I'm reading up on sums right now, and putting together plans for stuff to represent them in Python and Dafny. It's pretty straightforward to write Python code to evaluate concrete examples, but there are some wrinkles when it comes to writing stuff up in Dafny.

Stuff like:

I don't know whether Dafny has a rational datatype
A very very very quick skim of the Dafny documentation didn't reveal a way to define new semantics for existing operators... (And a slightly less quick skim indicates that this is explicitly not a thing you can do in Dafny.)
So an equivalent of sigma notation that may involve rational functions of the index values would seem to require some kind of usage of "traits".
Maybe it would make sense to write something that works in the integer domain to start with, and get some idea of the kinds of operations that I end up using?

Anyway, I'll hopefully finish skimming this chapter soon, and then go back to the beginning and read it more carefully. It's sort of interesting how trying to take an "implementation" perspective causes me to notice things that the text doesn't focus on. Like, if you're working with numbers mentally, going from the integers to the rationals is often not that big a deal (unless you're doing something with raising things to the powers of natural numbers, and that's the bit that gets changed). In software, that can be scary.

Anyway, I don't want to stay up any longer, so I'm going to wrap things up abruptly.

Good night.

Weekly Roundup 2024-01-16

2024-01-16T05:00:00-05:00

Wednesday: I didn't have a good time going back into work.
Thursday: I tried using marimo and Dafny to verify stuff related to Concrete Mathematics. It didn't go great to start with.
Friday: I got Dafny working much better.
Saturday: Too much work. (Remember that I date these the day after I write them...)
Sunday: I ran into some issues with Dafny; I basically had to declare a helper function "closed form" by fiat, even though it's not, I don't think.
Monday: I took a look at the harder problems from Concrete Mathematics, and decided, you know what, I don't need to do those right now.

Next week, I'm going to keep up with that, try to work on lore for my wife's writing, maybe get back to my own writing, and maybe get a better handle on audio editing because I have some weird ideas to get out.

Coding 2024-01-15

2024-01-15T05:00:00-05:00

Hm. I'm about to the first set of exam questions in Concrete Mathematics, and, hm. Wow. I'm going to have the smiley do a dramatic reenactment of what probably would have happened if I'd ever gotten questions like these on an exam.

:) (glitchier)

What?

What‽

Breathing too quickly...

World... going faint.

I hadn't realized how poorly that would translate from my thoughts to text, but oh well.

Looking over the other problems, these are all pretty intimidating. Maybe I should move along for now...

In any case, it's late and I should get to bed.

Good night.

Coding 2024-01-14

2024-01-14T05:00:00-05:00

Hm. I'm a little further along representing the Josephus problem in marimo and Dafny, but I'm not sure that what I have really constitutes a closed form.

I didn't figure out a way to use the standard library for this, so instead I ended up writing a helper that zeroes out the lower digits. If I can figure out a faster version of that, then I'll be happy, but I think for now I'll move on.

Also, I continue to thank Dafny for just how much math it seems to be able to do.

In any case, I think I managed to read the text closely enough to move onto generalizing it some. Let's see how that goes.

...

Hm.

I'm at the end of the section, and I haven't got code for this last example. Actually representing it looks like a bit much. I need to have d values of beta and d - 1 values of alpha, along with a value for c.

Whatever, I'll figure that part out later. For the moment, I can take a look at the exercises, and go "oh, past me did most of these, so there's really not much to be done here".

But really for the moment, I should get to bed.

Good night.

Diary 2024-01-13

2024-01-13T05:00:00-05:00

Good news: new toy arrived in the mail today.

Bad news: epically long day at work today, so I ended up too exhausted to set it up. I'll have to do it in the morning.

Nothing else worth talking about; work was massive.

Good night.

Coding 2024-01-12

2024-01-12T05:00:00-05:00

Things are going... very well with Dafny. I've got a general form that I'm doing, and I don't know whether it's idiomatic, but it seems to work for my purposes.

Basically, I define the recurrence relation I'm trying to solve stuff for as a function, define another function for the closed form if applicable, define a lemma that just handles the recursive case because the base case has been trivial every time so far, and write a method that is required to satisfy the recurrence relation, but is implemented by invoking the closed form, and passing the result to the lemma.

That was all pretty abstract. Let's try to make things a bit more concrete.

..smiley:

Heyo!

... Suppose we want to verify a closed form equation for the number of regions we can divide the plane into, given n straight lines. To start with, we need the recurrence relation, and the method that implements equivalent semantics:

function regions_from_lines(n: int): int
  requires 0 <= n
{
  if n == 0 then 1
  else regions_from_lines(n - 1) + n
}

method RegionsFromLines(n: int) returns (regions: int)
  requires 0 <= n
  ensures regions == regions_from_lines(n)
{
}

Should you be using nat?

I don't know, maybe? Probably?

Anyway, I'm conceptualizing this as the general template that I'm going to try to fit stuff into. Obviously, it doesn't currently verify, because I didn't write the code. Let's go through all of the steps, including the intermediate bits.

function regions_from_lines(n: nat): nat
{
  if n == 0 then 1
  else regions_from_lines(n - 1) + n
}

method RegionsFromLines(n: nat) returns (regions: nat)
  ensures regions == regions_from_lines(n)
{
  regions := 1 + n * (n + 1) / 2;
}

Technically, I could do a return statement here, but I'm going to stick the lemma afterwards, so this is easier to write later. Now, this, of course, does not verify, because Dafny isn't going to proactively try to validate the Mystery Formulas I give it. The next step, then, is to create an inductive lemma that validates the relationship between the recurrence relation and the Mystery Formula. Because this lemma will involve the Mystery Formula in its requires line, I'll define it as a function so I don't have to get it exactly in sync with itself:

function regions_from_lines(n: nat): nat
{
  if n == 0 then 1
  else regions_from_lines(n - 1) + n
}

function closed_form(n: int): int
{
  1 + n * (n + 1) / 2
}

lemma RegionInduction(lines: nat, regions: int)
  requires regions == closed_form(lines)
  ensures regions == regions_from_lines(lines)
{
}

method RegionsFromLines(n: nat) returns (regions: nat)
  ensures regions == regions_from_lines(n)
{
  regions := closed_form(n);
  RegionInduction(n, regions);
}

This moves the verification failures from the method into the lemma. At this point, I should probably emphasize that I haven't gotten any help or guidance from another person on this currently, so I could be putting together something that is excessively verbose, or inefficient, or just somehow hard to read. Regardless, the next step is to fill in the lemma with enough information for Dafny to carry out an inductive argument.

function regions_from_lines(n: nat): nat
{
  if n == 0 then 1
  else regions_from_lines(n - 1) + n
}

function closed_form(n: int): int
{
  1 + n * (n + 1) / 2
}

lemma RegionInduction(lines: nat, regions: int)
  requires regions == closed_form(lines)
  ensures regions == regions_from_lines(lines)
{
  if 1 <= lines {
    RegionInduction(lines - 1, regions - lines);
  }
}

method RegionsFromLines(n: nat) returns (regions: nat)
  ensures regions == regions_from_lines(n)
{
  regions := closed_form(n);
  RegionInduction(n, regions);
}

And, assuming I copied everything over correctly, we're done.

This seems like cheating somehow. You just ignored the base case, and the inductive step is just running the recurrence relation backwards?

It turns out that, once you've figured out which water to lead Dafny to, it will drink.

Even so, I feel like being skeptical. All you've shown us is that some layouts of file verify, while others do not. This could all be some kind of souped-up regular expression.

"Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?"

Okay, let's see what happens if I sabotage the closed form. Comment out the denominator, and... Hm. I'll have to see later if there's some way to make Dafny go into more detail, but it does catch the issue.

Anyway, with all that said and done, I've got some cleanup to do to the code(?) I've written, and then it's time to tackle the next section of the textbook. Very carefully and deliberately.

For now, better wind down and rest.

Good night.

Coding 2024-01-11

2024-01-11T05:00:00-05:00

I had this whole neat idea about doing exploratory coding in marimo, and then verifying ideas using Dafny, but my initial attempts in this area have so far gone... not great.

Marimo is fine so far, but when I gave Dafny a recurrence relation and closed form for the optimal number of moves for a tower of Hanoi puzzle, I expected it to either get something à la "Even a computer could discover this", or to quickly bail out because I gave it almost nothing to work with. Instead it just... sat there, with no indication of what it was doing, or how long it would take.

Just checked, and it's at least possible for Dafny to successfully verify a file, so it's not obviously an issue that occurs every time.

After a little more poking, this does have something to do with the method specifically, but I'm, like, not sure what to do with that yet. I'll have to think about it, I guess.

...

Hold up, I searched the issues for some key words, found stuff that eventually led me to the experimental --disable-nonlinear-arithmetic flag. I pass it in, and...

Yay! It fails!

Yay?

I needed it to commit to a course of action.

So, given that verification is failing, I think I need to write a lemma, and have it work via induction? I'll try to figure this out later. For now, I want to sleep.

Good night.

Diary 2024-01-10

2024-01-10T05:00:00-05:00

First day back in the office. Uuuuuuuuuugh.

I've got a few things I want to try to look into, but it's pretty hard to work up the energy for that when the sun's down by the time I get home. The main thing is trying to find a good mind-mapping tool, because I tried to map stuff out about my writing in a notebook, and the result was totally unreadable. Another is getting back into Concrete Mathematics, and trying to work on the exercises in marimo. Hopefully, I'll be able to transfer stuff there to calendars for worldbuilding, and balancing incremental games.

Anyway, I need to get ready for bed.

Good night.

Weekly Roundup 2024-01-09

2024-01-09T05:00:00-05:00

Wednesday: We got home, and I got a few small things done and started.
Thursday: I talked some about a project I've been working on in various forms for a while now.
Friday: I continued to work on that project.
Saturday: Nothing much happened, and this frustrated me.
Sunday: I got a few things done, and called that a win.
Monday: I got a bit more done, and then took it easy.

Next week, I'm going to probably be messing around with writing projects, but we'll see.

Diary 2024-01-08

2024-01-08T05:00:00-05:00

I finished up with the list I put together for yesterday, and then took things easy. I guess we'll have to see what starting work back up tomorrow feels like; probably really disorienting.

For the moment, I'm tired and having trouble writing, so oh well, cutting this short so I'm not obsessing over it all night.

Good night.

Diary 2024-01-07

2024-01-07T05:00:00-05:00

Well, that was an interesting mixed bag. After the last entry, I put together a list of things I want to accomplish soon. I got the stuff that I considered highest priority done, so I guess today was successful. The rest of the list should be workable to take care of tomorrow morning, so I'll get that done when the time comes.

One of those things was finally reading the stuff I wrote to my wife. Got some positive feedback there. I'd like to write some more, but my current feeling is that I should try to take all of the names and people I made up, and figure out how they fit together. Stuff in the text and explicit speculation only; not going to write lore.

Anyway, I let things get late, which is part of the reason I didn't do the lower-priority stuff while I was putting this entry together. It'll all be fine.

Good night.

Diary 2024-01-06

2024-01-06T05:00:00-05:00

Every day since we got back, I've thought "Well, I'll get a bunch done today." and it hasn't happened yet.

So, fine, I'm going to lower my expectations and/or perform reverse psychology, and declare that I'm going to take things super easy tomorrow, and see how that works out!

Good night.

Diary 2024-01-05

2024-01-05T05:00:00-05:00

I didn't really have "momentum" today, but I did manage to do a trial read of the writing using YSK. It's mostly fine, but one thing I wasn't expecting is that using these words that aren't part of standard English caused my pronunciation of the letter R to shift? Like "These words aren't English, so I'd better pronounce things like some other language?" I guess...

I don't have anything else in mind to talk about, so I'm just going to post and wrap up.

Good night.

Diary 2024-01-04

2024-01-04T05:00:00-05:00

Seems like I'm not ramping back up in any impressive way. I've got some writing that I'm revising. That's not ready until I've read it out loud to check for issues with how it flows. At least it's not metrical poetry, so I'm not going to be sounding it out and going "Well, formally, this fits the meter, but I can't actually pronounce it with that emphasis."

I might as well talk a little bit more about this project. From an inspiration perspective, I've got some inspirations, and things that I think I should read for inspiration, that I think sound pretty impressive, and I suspect that the result will be... not. I think it could speak to some people, but this kind of feels different from other projects, where a lot of the time I could imagine it getting a lot of traction and influence—

If it came out decades ago into a world where its own inspirations somehow didn't exist.

Yes, yes, that.

But the point is, this project, which I haven't properly named yet, feels like it has a much smaller ambition. I don't want to talk too much about the content; I'd rather let it speak for itself when I'm comfortable releasing it. However, I can talk about one of the reasons I'm putting effort into something that's intentionally "smaller".

This writing, and some other writing I've done in the previous year (some of it more on the ambitious side of the scale), is meant as a vehicle for... I guess I'll call it an experimental variant of English? Swap out three or four basic word roots for, let's say three new roots, doing the work of six.

The point of all of this, the entire reason, the thing that, if you don't understand it, will render everything I am doing here incomprehensible

Pay attention, this bit is important.

is that one of the major obstacles I've encountered in writing over the past few years is feeling slowed down by the fact that I have to commit to a character's gender identity if I want to use pronouns for them, and be able to use multiple pronouns to be able to differentiate between them. I know this problem probably won't resonate with a lot of people, and I don't always have it, but I have it enough that I just wanted a break.

So, I devised what I'm calling the "YSK paradigm", named after the first letter of each of the roots for the new pronouns. I don't feel like going into detail about spelling/pronunciation right now (there's a non-zero chance I'll end up revising the current version), but I'll lay out the basic idea:

The "Y" pronouns indicate "this person or people"
The "S" pronouns indicate "that person or people"
The "K" pronouns indicate "that person or people over there"
There are singular and plural versions.

Now, I've failed to explain this before, and in an attempt to anticipate that failure mode, I'd like to say, if one of the thoughts you have after reading the last few paragraphs is "If you're using these, it would help to have a table beforehand explaining which characters get which pronouns", then you have gravely misunderstood me, and if you wish to rectify that, you should reread from after the smiley box.

"smiley box"?

Well, what's anyone supposed to call that thing ↑?

Anyway, maybe I'll have stuff using these put out publicly soon, maybe not. We'll see.

For now, I once again need to get to bed.

Good night.

Diary 2024-01-03

2024-01-03T05:00:00-05:00

All right, we're back. Time to relax a bit.

I found the color stuff I was looking for; it was Reasonable Colors.

Aside from working that stuff out, I didn't get too much done today. I discussed some lore for a project my wife is working on. We made some important progress there, but nothing that would look big if we pointed at it, if that makes sense.

And now it's late, and I'd like to get things wrapped up in the next few minutes.

Good night.

Weekly Roundup 2024-01-02

2024-01-02T05:00:00-05:00

Wednesday: I made some progress with MOTR, at least in terms of planning changes.
Thursday: I executed the plans, and ran into issues, which I eventually fixed.
Friday: I got to a point with MOTR where I need a break; I may have some plans to look into later.
Saturday: I took things easy.
Sunday: I complained about Vue.js. Oh well.
Monday: I explained what I was looking into Vue.js for, and how I'm looking into restricted color palettes.

Next week, we're heading back home, then I'll work more on writing or something.

Diary 2024-01-01

2024-01-01T05:00:00-05:00

Okay, let's see. It's kind of late now, but we'll probably be up for a while longer. Thinking about the stuff I was talking around yesterday.

I've got a few ideas for incremental games that I want to mess with, and I think that The Modding Tree would be a good fit for them in terms of functionality, but. The visuals make my head hurt. Too much contrast, too small text. Sometimes far too little contrast. From a layout perspective, things can get really janky.

So, I want to try to rethink a lot of the visual language, and part of that is restricting the color scheme so that there's more "accent colors" and less "the color of this layer is an animated gif". I'm thinking of doing this based on Selenized, but I was also thinking about trying to use a color palette that I'm having trouble finding, and it's bothering me...

We'll see if I ever remember, but for now, I want to shut things down now.

Good night.

Diary 2023-12-31

2023-12-31T05:00:00-05:00

I could have made this into a coding entry, but I didn't. The main thing worth talking about that I did today was to take a VueJS 2 project that I'm passingly familiar with, and try to create a fork using VueJS 3. I very, very quickly ended up out of my depth, and decided, you know what, I'm going to focus on the other things I want out of this project.

(The problem I ran into was how to convert set and delete calls into uses of proxy objects, when I was having trouble tracking down which code actually populates the values that need to be replaced with proxies. If someone can point me (on Mastodon or Cohost or whatever) to a resource that explains how to deal with this in an unfamiliar codebase, I think that's all I'd actually need to attempt the update again. No advanced libraries or anything complicating the migration.)

Anyway, the more significant thing I wanted to work on with this was a comprehensive overhaul of the visual language, because it's kind of a bad sign, I think, when I have to squint at small numbers on weird contrast backgrounds that give me crazy afterimages.

I'll try and explain what I'm on about later; I want to wrap things up right now.

Good night.

Diary 2023-12-30

2023-12-30T05:00:00-05:00

Hm. Not too organized today. The most significant thing I did was some research for a project I may write up here, we'll see.

Aside from that, I'm futzing around with games to maybe play. Maybe later, I'll play them, and maybe I'll write something about them. We'll see. Maybe that'll wait until I have access to a controller again.

I think I must be just recovering from some of the issues I ran into trying to update MOTR. We'll see what I'm up for doing in the next few days.

For now, I really should get to bed.

Good night.

Coding 2023-12-29

2023-12-29T05:00:00-05:00

Okay. Okay. Okay.

I've finished up the current round of code reorganization, and this has successfully excised that one branch I didn't want to write tests for. Good.

Unfortunately, this has left the code in a state of... chaos would be a strong word. Basically, I've been isolating the code into logical units based on how different pieces of data relate to each other, and I've basically ended up with more of these logical units than I have concepts.

Believe it. One of the new classes is literally just named Foo.

So, obviously, I would like to address that. But I've got plenty of experience from while I was doing the recent rewrites, of trying to do just that, and bouncing off. Now, hopefully this will be easier following the rewrites, but I think for sure I need to take a break and look at this with fresh eyes.

I've also got some out-there ideas that might not work, which could simplify things a little, potentially. (Initial simplified tests indicate that it could help. The idea is to try to cut down on the absurd proliferation of type variables, by, um, creating more type variables.) Here's the playground code I wrote so I hopefully remember what I was thinking of doing:

from typing import Any, Generic, TypeVar

L = TypeVar("L", bound=list[Any])
T = TypeVar("T")

class Hello(Generic[L]):

    def fst(self: Hello[list[T]]) -> T:
        raise NotImplementedError

Anyway. I can try to get into other coding projects, or I can get back to some writing that I kind of left myself hanging on. We'll see.

For now, though, it's not worth trying to do any more tonight.

Good night.

Coding 2023-12-28

2023-12-28T05:00:00-05:00

Oh boy. I've gotten so close to getting this stuff working, I just need to figure out what I did to make Mypy unhappy with the code related to — Aw man, I figured it out, that was awful.

See, I've got GenericRunner, which has specializations to Runner and ModuleRunner, and doing that was perhaps not the best idea, because of stuff like "accidentally passing an unbound method reference scoped to Runner instead of GenericRunner". Ugh, oh well.

With that bit of insight obtained, I've gotten Mypy to accept the required upstream changes, and gotten the tests to pass. Once I have the many flake8 issues that I have doubtless introduced addressed, I can finally make the changes I intended to make in the GenericRunner class itself, and see how that shakes out. Once those are in, it's one more round of renames, and then I can merge this to the coverage topic, then have a completely awful merge up to the future topic.

Wish me luck...

Anyway, that's all happening later. Right now I'm going to get to bed so I'm not up super-late again.

Good night.

Coding 2023-12-27

2023-12-27T05:00:00-05:00

Hoo boy... I just roughed out the Protocol for handling the different behavior needed for ParametricCommand stuff, and keeping straight what it does is going to be a nightmare. I'll try to run through it quick before I get ready for bed. Basically, there's one method that takes a Matrix, and may modify it.

If the method is called on the newtype-style wrapper, it must pass back its argument, unchanged.
If the method is called, specifically on a ParametricCommand[Module, _program.T, _installer.T, _runner.T], it must accept a Matrix[ModuleRunner[_entry.T, Any, _installer.T], and return a Matrix[GenericRunner[_entry.T, _program.T, _installer.T, _runner.T]].

Strictly speaking, it probably doesn't have to be quite so detailed with the types, but every Any makes it harder to think about. Either way, I'll see if I'm up for implementing all this tomorrow.

Good night.

Weekly Roundup 2023-12-26

2023-12-26T05:00:00-05:00

Wednesday: Bad time management.
Thursday: I messed around with a few things, but nothing in enough detail to write about.
Friday: I started tinkering with MOTR again.
Saturday: Bad time management.
Sunday: Decent time management, but due to factors, I still didn't have anything to write about.
Monday: I got some insight into changes I'm going to want to make to MOTR.

Next week, I'm going to try to get stuff done, now that we're not rushing around.

Coding 2023-12-25

2023-12-25T05:00:00-05:00

Okay. So. I'm in the middle of some kind of big weird rewrite regarding what used to be CommandBuilder and is currently Runner, because that name's not great, but it's better.

The actually important change is that I added a parameter to Runner to replace one of its Optional fields. The upshot of doing that locally is that I get to delete a conditional branch that was really bothering me. However, actually going through with this change incurs a lot of costs, and I figured I'd try to pay those costs before getting too attached to the benefits. Currently, I'm squirreled up in the parametric_command code.

Let's quick lay out what's going on there, because if I just try to touch the code without planning, I'll probably mess it up. So, there are two axes of variation I care about. They're on the ParametricCommand class, which has two fields: a ParametricCommandMeta[...], and a ParametricCommand[...] | SelectionDirectory[...].

Now that the ParametricCommandMeta class has a _runner.T parameter, it's necessary to look at how the actions there interact with the union members. The answer is, that if _runner.T is None, then the union can't be a ParametricCommand[...], because that branch of the union represents something being run as a module.

Now, the interesting question is whether the opposite case should also be disallowed. I'm going to go with a firm maybe: I should go with whatever makes the implementation easier.

And speaking of the implementation, if I'm going to change this union into something that keys off of the _runner.T, I'm going to need to write a parametric protocol to handle that, and to make that work, I'm going to need to write a wrapper class for the SelectionDirectory. Then, if I've got a protocol anyway, I might as well leverage it by rewriting all of the ParametricCommand methods that currently dispatch on the union branch into protocol methods.

Key note for my future self: because the wrapper class is going to basically be the newtype pattern, I don't need to explicitly expose the class to the rest of the code; I can just apply it in the with_directory() function, keeping the calling interface the same.

I think that covers everything I was thinking about. I'd try to do it now, but, as usual when I'm writing these, it's late.

Good night.

Diary 2023-12-24

2023-12-24T05:00:00-05:00

As anticipated, there was a bunch of travel today that kind of got in the way of having much to write about. I have made some minor progress on a few things, but nothing yet worth discussing.

The travel led to a good time, but being on the road for so long, it's hard to deal with it.

I'm not thinking too clearly, so I'm going to try to get ready for bed soon.

Good night.

Diary 2023-12-23

2023-12-23T05:00:00-05:00

I ended up not having much today to write about. I'm going to be on the road a bunch tomorrow, so I'm not going to have much tomorrow, either. These next few days are going to have a lot going on, so now I'm thinking about, once I'm through all of that, I'm going to need to figure out how to actually get stuff done.

I'm done for now.

Good night.

Coding 2023-12-22

2023-12-22T05:00:00-05:00

I did a few things today, but one of them was picking MOTR back up from where I left off. This basically boiled down to fixing all of the Mypy and other errors caused by removing a single default value.

If I remember what I was thinking when I took a break, the next step is to consolidate the field that I changed with the one after it, into yet another new data class. Then I can add a new type parameter to the class that still needs to be renamed, and then I get to fix a new set of Mypy errors. I remain hopeful that this will all resolve into an improved layout and data flow, but it is hard to deny that the process of getting to wherever the endpoint is, kind of looks like someone deconstructed all of a multi-course meal...

Anyway, I need to wrap things up and get to bed.

Good night.

Diary 2023-12-21

2023-12-21T05:00:00-05:00

So, I've got more time tonight, and I have something to write about this time.

I've been messing around with writing really short-form stuff in a vague ambiguously-fantasy setting. I'm trying to draw from the early works of Lord Dunsany, from Invisible Cities, and from Tlön, Uqbar, Orbis Tertius.

I feel like this is kind of hemming in some of my worldbuilding-related excesses, because the "point" of the setting is to raise questions, more or less, so it doesn't help me any to try to "stockpile" answers.

There's another point to all of this writing, but I don't feel like I can explain it coherently right now. I've tried to explain it to a few people in the past, and got frustrated when stuff that I thought was completely clear just got, like, glossed over.

But I shouln't focus on that right now, because the main thing I'm feeling right now is tired.

Good night.

Diary 2023-12-20

2023-12-20T05:00:00-05:00

It's late.

I did unimportant stuff for most of the day.

I have less than no time to write this, so I'm not going to.

Good night.

Weekly Roundup 2023-12-19

2023-12-19T05:00:00-05:00

Wednesday: I didn't have much to say because I was distracted. Miscellaneous bits of work.
Thursday: I rewrote some code for representing playing cards. It had issues.
Friday: I found some things that marimo never claimed to be suited for, and definitely isn't.
Saturday: More progress on the card code, and then I decided to step away from it until I figure out what it's for.
Sunday: I made some minor progress writing.
Monday: Terrible time management.

Next week, I'm going to hopefully get more on top of all of these projects. We'll see.

Diary 2023-12-18

2023-12-18T05:00:00-05:00

"Haha, why don't I play some silly browser games and oh no it is 11 PM."

Anyway, I also found some good inspiration stuff for my writing; I'll talk about that later. For now, I need to salvage things by getting into bed as soon as I can.

Good night.

Diary 2023-12-17

2023-12-17T05:00:00-05:00

I did some short-form writing that I think it helping me with the stuff I intend it to help with. This stuff is all so short that I should probably try to figure out some way to turn it into poems, or better-structured "flash fiction".

I let things get late, and I don't have anything else to say on the matter, really. Good night.

Good night.

Coding 2023-12-16

2023-12-16T05:00:00-05:00

The card code is shaping up a bit. I thought I was going to change the "design" somewhat, but I ended up just improving the implementation of the existing interface. I added some helpers and a new type, and then created a bunch more decks.

I think I should try to change gears from this for a while, but when I come back to it, I should probably try to figure out how to write pretty-printers for decks.

Anyway, I'm going to stop thinking about that, and try to focus on writing instead. I've got some stuff I need to focus on, but some of the details there are flexible. Anyway, I've been spacing out listening to utterly cursed song remixes for too long, so I'm just going to cut this off abruptly.

Good night.

Coding 2023-12-15

2023-12-15T05:00:00-05:00

Okay, let's see...

I was looking into doing Tkinter prototyping in marimo. I think I would not recommend this; I anticipated the obvious caveats, but there were some non-obvious issues, that I'm not sure what to think of. (Basically, I wanted to put the mainloop() calls in a UI callback so there's like, a "setup" phase, and then I can trigger the actual window stuff with a button. This doesn't work, but I don't know why it doesn't and if it "should".)

Anyway, I messed around with the card code as well, and I'm stuck against, like, I need to take this rough prototype and then properly think through how it should all relate. Even with some of my attempts to clean stuff up, it's still a bit of a mess. I'm tired now, so I'm going to wrap up and try to figure out the design I should be aiming for.

Good night.

Coding 2023-12-14

2023-12-14T05:00:00-05:00

Oof, partied too hard today, in a way that was imperceptible at the time. Let's see how much I'm up for.

I dusted off some of my old hobby code today, copying it from a Jupyter notebook into marimo. Along the way, I made some changes. See, the original version had a bunch of type-level stuff going on, and I ended up replacing what I was trying to do there with runtime validation. This was mostly fine in my opinion, but I think I swung the pendulum slightly too far, and I need to find some way to recover the marker-y nonsense I had going on, to a limited extent.

Anyway, the code itself. I can't explain why, but I wanted to have a minimal toolkit for expressing arbitrary decks of cards. Currently, I've got code that can generate a normal 52-card deck plus any number of jokers, and both Japanese and Korean hanafuda/hwatu decks. My assumption is that that's enough flexibility to handle most traditional decks; I'm not intending to handle anything that has a regular release schedule, for example.

So, what's left to do before I ponder trying to library-ize this code?

Get some type-marker-y stuff in, and tighten up some annotations.
Figure out how to pretty-print card names, and generally how to write multimethods against them. Right now, I'm using the default attrs methods for printing, which are way too verbose for most purposes, but any replacement is going to need some form of contextual information that I'm not sure how to provide.

Anyway, I'll think about that, and other things, and try to make it to the weekend.

Good night.

Diary 2023-12-13

2023-12-13T05:00:00-05:00

I wasn't really organized enough to have a single post to write today. I've been reading a bunch, and I should be done with that book soon. In any case, I messed around with a few other things.

I made some further attempts at musical arrangement, and the result is... it sounds okay so far in aggregate, but it becomes much weaker when broken down to its basic components, which bothers me somewhat.

I also did some worldbuilding stuff, and I'm making some progress there. The specific stuff I'm working on, I'm probably not going to pick back up, so I'll try to switch focus there to some projects that are more relevant to me.

For now, though, I really need to be getting to bed.

Good night.

Weekly Roundup 2023-12-12

2023-12-12T05:00:00-05:00

Wednesday: I investigated the basic sounds of some common chord loops.
Thursday: I made a little more progress with that, but not much more.
Friday: I fiddled around with marimo, and had some weird ideas about stuff I could try to do with it.
Saturday: I wrote some very normal Python code. There is no need to look at the post to confirm this.
Sunday: Traveled out, didn't have much to say.
Monday: Traveled back, had slightly more to say, which still wasn't much.

Next week, I've got a few things lined up. I'm attempting to arrange some simple music for, um, nine parts... And I've got some stuff ready to mess around with for some worldbuilding from months ago... And I've got some other worldbuilding I'm trying to put together for some stuff my wife is working on.

Diary 2023-12-11

2023-12-11T05:00:00-05:00

Travel back.

I did research on a few things I was interested in, and I think I'm getting a better handle on them, but I don't feel like I have anything interesting to say about them yet.

One thing: as far as music goes, I'm going to try arranging existing pieces for combinations of instruments I'm interested in, and see if I learn anything from that. As far as that goes, I feel like I just need to do some minor research and thinking to handle most of the instruments, but I don't know anything about arranging for percussion, so that's going to be a thing.

I do have some work I want to do on other things, but I don't know if I have the energy to talk about it now, so I'm not sure about doing it. But, I'm spacing out right now, so I'd better wrap up.

Good night.

Diary 2023-12-10

2023-12-10T05:00:00-05:00

More travel today.

I'm still pondering ways to push my usage of marimo, but I don't have anything to show for that yet. Hopefully I'll have something there in a few days.

Aside from that, tonight was a lot, and I'll just hurt myself if I push really hard. It's time to chill out.

Good night.

Coding 2023-12-09

2023-12-09T05:00:00-05:00

Still doing stuff in marimo. The bug I reported got fixed, which I'm glad about. It sounded like it wouldn't be too hard to address, given familiarity with the code, and I think that's how it ended up.

...

Hm, yeah, looks very much like a "small change, but you have to know which small change".

Aside from that bug, I wrote some extremely normal code.

:) (glitchy)

You have to show them. It's hilarious.

You know, there are languages where this functionality would be completely unremarkable.

:) (glitchy)

I want you to make them imagine the calibre of Mypy plugin that would be required to make this even vaguely okay.

It only raised one error in the playground!

...

Fine.

class GenericEnumType(enum.EnumType):
    def __getitem__(cls, key):
        if isinstance(key, str):
            return super().__getitem__(key)
        return cls.__class_getitem__(key)

:) (glitchier)

yes... ha ha ha... YES

I will say that I really do appreciate how marimo made it easier to prototype these Metaprogramming Crimes. Just regenerating everything downstream when I want to quick try out another approach. That would be a nightmare in a REPL or (to a somewhat lesser extent) in Jupyter.

Although, on that note, I'd kind of like to apologize to the marimo team for taking the capabilities they gave me, gave everyone, and doing... that.

Yeah, how do you sleep at night?

Soon.

Good night.

Coding 2023-12-08

2023-12-08T05:00:00-05:00

The inscrutable exhortations of my soul told me to transcribe mathematical functions from a book into a marimo notebook, so I guess that's what I did today. I'm right now at a point where I need to decide how I want to represent some numbers; this is likely to result in a significant amount of thought for very little change in the behavior of the code, and I'm at peace with that.

I am, for reasons, not going to go into much more detail about this code, so let's see if anything else occurs to me...

Here's an idea that might be questionable: with the right design, an interactive fiction library compatible with Python could use marimo to do output validation similar to the "Skein" concept from Inform.

...

So, I just looked into interactive fiction in Python, and nothing leapt out at me as "this is well-suited for what I was thinking of". I'll just let this idea stew for a bit, and focus on other things for the time being.

For the immediate future, though, oof, sleep.

Good night.

Music Theory 2023-12-07

2023-12-07T05:00:00-05:00

Still messing around with basic stuff. I think I need to get a little bolder with my melodies, since the stuff I had before was pretty much all stepwise motion.

There is a major obstacle to self-directed stuff here, and that is that my ability to judge how well something works seems to be hampered by a deep sense of satisfaction as a result of "hear one thing three times, then something else, then repeat from the beginning". Maybe I can build that up somehow?

I'll have to think about it later, because it's once again really late and I'm tired.

Good night.

Music Theory 2023-12-06

2023-12-06T05:00:00-05:00

Wow, the last post in this category was a while ago. Anyway, I started small with this whole "music" thing. I came up with some rhythmic ideas I thought were interesting, put them over basic chord loops, found stable-enough scale degrees to put in the melody, and ended up with minimal cores of what could potentially become a song. The thing I need to deal with is just how much heavy lifting the choice of chord loop seems to do:

Royal Road progression → sounds like it could be from some random OVA I saw ten years ago and forgot about.
The progression that's been common in pop music → a pop song.
The same progression, but in minor key → a pop song, but it's sad now.

It's kind of weird to think about how much saying "these chords in this order" seems to constrain what the song sounds like. Like, does it end up with "choose the chords to set the tone, put the desired melody on top, and then just puzzle-game the voicing into place"? Is that how this works? I must be erasing some vital nuance here.

Anyway, I'll leave that to tomorrow, because it's late and I meant to already be in bed.

Good night.

Weekly Roundup 2023-12-05

2023-12-05T05:00:00-05:00

Wednesday: I came to a decision in MOTR that I didn't want to rush, so I'm going to be focusing on other things for the duration of this roundup. I thought about looking into some programming languages besides Python. (And then mostly didn't.)
Thursday: I dusted off one of my old projects, and immediately got confused by the Python standard library.
Friday: I figured out what was going on with the stuff I was confused by, but I was really tired, so, not much happened.
Saturday: Even more tired.
Sunday: I switched "projects to try working on in marimo" again.
Monday: A bit more marimo, including a very specific bug report. I resolved to back-burner the "algorithmic music" stuff I was aiming towards, in favor of getting more non-algorithmic experience.

Next week, we'll see what I can manage there.

Coding 2023-12-04

2023-12-04T05:00:00-05:00

Okay, I've been poking at a few marimo notebooks today, (and filed another bug; don't worry about it), but I've ended up in a frame of mind again where I want to totally overdo the architecture of music-related stuff. Maybe if I describe what I'm thinking about there, it'll alleviate the urges somewhat.

So, background. Historically, the intervals between different notes were rational numbers with small-enough numerators and denominators; this basically equates to "sounding consonant". Using rational numbers for this purpose has some good qualities, and some qualities that made it hard to compose certain kinds of music. In the western musical tradition, this system was overtaken by one based on the twelfth root of two. This greatly simplified composition and tuning, though it did result in some intervals sounding off. However, they're consistently off, so that's good enough, especially now that people are used to them.

Now, that's all well and good if you're not a huge nerd, but I'm over here wondering about stuff like "But what would it sound like to use one of the old systems?" (Yes, systems, plural. There were varying ideas about "the right way" to choose rational intervals.) "In the context of roots, what if we use a different root than twelfth, or a different base than two?"

There has already been a lot of investigation into this, and a bunch of existing software. From a modeling perspective, it would be "nice" to be able to track intervals in terms of fractional powers of primes, but actually doing all of this is a bunch of effort that other people have figured out the right way to go to.

While I was writing that paragraph, I had a look at the tools page on the Xenharmonic wiki, and tried some of them out. With what I've found in the course of a few minutes, here is what I've decided is the sensible thing to do:

Focus on "traditional" (twelve equal divisions of the octave) stuff at first, in whatever tools feel good.
Figure out what it takes to replicate the basics of such compositions in a tool that supports xenharmonics.
Start tweaking things to make use of less traditional tunings.

That's, like, effort and stuff, compared to trying to come up with weird algebraic representations that only need to work in theory, but it really would be better to focus on practical matters.

Later, because it's late now and I'm tired.

Good night.

Coding 2023-12-03

2023-12-03T05:00:00-05:00

I'm still messing with marimo for, I'm going to call it "sketching" with Python. It's working out well for that. I decided to change focus from [NOT VERY EXCITING SECRET] to Music for Geeks and Nerds. I'd previously messed with it, or possibly tried to reinvent things from first principles, but those efforts were plagued by overthinking. I'm going to see what I can accomplish by just following along with the book, and "sketching" code rather than putting together elaborate scaffolding.

Anyway, if I'm honest with myself, I'm actually pretty tired, even if it's "early" by my usual standards, so I'm going to wrap things up and try to get a reasonable amount of rest.

Good night.

Diary 2023-12-02

2023-12-02T05:00:00-05:00

I was pretty out of it today, so I'm not going to push myself. This is in part (or maybe a lot) because (I'm pretty sure) I haven't been getting enough sleep lately, so, like, I should get to work on that.

First step on that: start moving towards bed earlier. I'm already super tired, so, like, if I keep up, I'll be really unhappy. Okay, that's enough justifying my decisions.

Good night.

Coding 2023-12-01

2023-12-01T05:00:00-05:00

I think I have a better understanding of the enum stuff from yesterday, and I can put things together more nicely, later.

All told though, my verdict on today is that I was pretty tired, and I managed to get some stuff planned out in spite of that. I can potentially get some stuff done before tomorrow, but I'm not going to force myself. I'll just see what I can manage while this entry publishes.

Good night.

Coding 2023-11-30

2023-11-30T05:00:00-05:00

I haven't really touched the languages I was talking about last time, but I have thought of something to attempt with them that should be more interesting than running through a couple of basic introductory exercises. I have a book, which documents some code, which I wish to adapt for purposes outside of its original design. I could try to work with the code in its original language, but I... don't want to. So, let's try working with it in those languages.

My plan is to try to express things in a marimo notebook for prototyping, and then try to transplant that design to the other languages. We'll see how that goes; when I was thinking about this earlier, I remembered running into Weird Problems because I was trying to customize Python's enum support in a very specific way that must not have been expected.

...

I tracked down the problem I ran into, and now this is just going to bother me. Basically, I was trying to make an Enum where the auto values start at 0 instead of 1, and the code has both:

The way to do this that seemed elegant and obvious to me
and the viscerally disgusting hack that I used when the elegant and obvious option had no clear effect. (Also included is light diagnostic code confirming the necessity of the hack.)

...

Found it.

v.value = self._generate_next_value(
    key, 1, len(self._member_names), self._last_values[:],
    )

:) (glitchy)

Why does the staticmethod take an argument if it's always going to be a constant?

I'm not sure why the interface is like that, but now I hypothesize that the "hack" is almost the right version, and I should remove the "elegant fix", because it's subtly breaking one of the interfaces. I'll see what I think tomorrow.

Good night.

Coding 2023-11-29

2023-11-29T05:00:00-05:00

My laptop hasn't done anything too objectionable since last time.

So far.

Anyway, I poked a bit more at MOTR, and it's going well, but I need to make a decision. There is an attribute that is currently optional, and I want to change the type from a union to a bound typevar. This is incompatible (currently? I'm not sure if this PEP would change that) with supplying a default value to attrs, so I need to make it a required argument, but it kind of sucks to make downstream stuff explicitly pass None, so I'd like to have some alternative constructors, and then I need to work out what those should look like. I'm somewhat leaning towards a minimal default constructor, and then "fluent" modification methods to open the type space back up, but this feels like something I need to think about, at least subconsciously.

So, I'm going to let that sit for a bit, and also ponder the other languages I'm considering looking at. The initial field was pretty weird and uneven, but right now it's narrowed down to Rust, TypeScript via Deno, and Kotlin. I'm not sure if the criteria I used were applied consistently or really make any sense, so please don't consider any of the choices I made or didn't make to be significant.

I'll have to look into doing something with them later. For now, I'm going to wind down.

Good night.

Weekly Roundup 2023-11-28

2023-11-28T05:00:00-05:00

Wednesday: I messed around a little with marimo, a Python project that wants to compete with Jupyter. I didn't do anything that really evaluated how well it competes, but I did find some minor rough edges, which eventually translated into a documentation update.
Thursday: I followed up on marimo, and then I got some messed-up behavior out of Mypy.
Friday: I fixed some Mypy issues by ignoring them. (And putting a bug URL in a comment.)
Saturday: More progress on MOTR.
Sunday: I kept on splitting up stuff in MOTR.
Monday: I continued on that, and things worked out well.

Next week, I'm going to see how much further I can take that. However, I finally put together some obvious conclusions about how to put together a tech stack for hobby stuff, so I've got research to do.

Coding 2023-11-27

2023-11-27T05:00:00-05:00

We had more travel today, and some technical issues with the laptop. However, I had time to start tweaking the class structure in MOTR, and it improved some code that I hadn't been thinking about, so that's a good sign. I'll see how far I can chase this, and then finally look for something else to focus on.

I just now got ready for some of the changes that will end up potentially being much more complicated, but they promise to eliminate two lines of missed coverage, so I'm going to consider them worth it, because I really don't want to cover those lines.

Anyway, I need to wind down, and recover from stuff like the technical issues.

Good night.

Coding 2023-11-26

2023-11-26T05:00:00-05:00

All right, after messing with other stuff all day, I am taking a closer look at the increasingly-ineptly-named CommandBuilder class, and I think I'm getting some ideas for how to go forward. (No, for whatever reason, I am not tired of this.)

The CommandBuilder has five fields, but the address a number of interlocking roles. First up, there's executable and partial_invocation, which are processed together to produce an InvocationEnvironment. Then, allowed_codes (which I recall is renamed in the future topic, but eh), pass_module_codes, and module_args.

There are a few things going on with those. One is that module_args is a Union, but I want to turn it into a constrained type variable. This would have some downstream effects that I haven't thought all the way through, but if I can put the distinction between "has module args" and "doesn't" into the type system, I want to. However, looking at these fields with fresh eyes, I see that pass_module_codes doesn't mean anything when module_args is None, so those fields should really be a single field that can contain a product type. Furthermore, the purpose of pass_module_codes is just to control the evolution of the allowed_codes variable, so it should be bundled together as well.

This works out to: ModuleInfo, CodeInfo, ExecutableAndInvocation, and...

And...

:) (glitchier)

Out with it.

I don't know! Somehow, using "command" in some capacity still seems to make sense, because this is the first level where it's looping back around to the scope of information in the task class, which is called Cmd. But keeping it feels like a copout. It's kind of like, these new names aren't great, they're fine, but not great, and bringing them together like this pushes them to the breaking point.

Well, I've thought about this for a long time now, and no luck. Maybe I need to just actually try it, either tomorrow or in a few days. For now, rest.

Good night.

Coding 2023-11-25

2023-11-25T05:00:00-05:00

Still tweaking MOTR and managing my time poorly. With a little more messing around, I should have the renames propagated all the way up the layers, and get back to imlementing wrappers. Sadly, even though I am making progress, it's slow going, and it looks likely I'll need a few more passes once I've got the module structure sorted out.

Regardless, it is way too late and I should get to bed.

Good night.

Coding 2023-11-24

2023-11-24T05:00:00-05:00

Well, as far as marimo goes, we agreed on a documentation change rather than a code change; it's not onerous to get the right behavior once you understand what can make the behavior wrong, but the factors that can make the behavior wrong are subtle. So, documentation.

Besides that, I cleared up some issues with MOTR. At this point, I either get back to working on the names, or I find some other project to make progress on.

I would have liked to get more done today, but Thanksgiving is always pretty disruptive.

I'm (still? I forget) trying to figure out if there's some better way to accomplish what I want from MOTR. The painful truth, though, is that I can't see how any of the alternatives could compete with the hacked-together motrfile that MOTR itself currently uses. So, it's like, it's the best at delivering what I want, but still not good enough.

I've reviewed the state of the code, and the obvious place to work on is where I've been focusing already, so, eh... I think I need to find something else to work on as a distraction, but we'll see how things go. Later. For now, I get to bed.

Good night.

Coding 2023-11-23

2023-11-23T05:00:00-05:00

Update on marimo: one of my concerns about it was unfounded, and another is intended to be worked on. Here's the quick summary: do not use from __future__ import annotations, because marimo code implicitly has it. There are a few rough edges around this, but they're working on it. I haven't tried using anything like Pydantic in this environment, and I'm not sure what would happen, so that's the main concern I'd have about usability.

I took another look at MOTR, and found some more issues I need to fix in the plugin, but then I ended up with some issues that confuse me, and I think I did something wrong outside of the plugin. It's hard to describe what's really going on, besides "It's pretty weird", so I'm going to have to sleep on it.

I'm going to do some writing while this publishes.

Good night.

Coding 2023-11-22

2023-11-22T05:00:00-05:00

I ended up not focusing on much today. Mainly, I messed around with some of the tech I was interested in messing with. Specifically, marimo.

As I understand it, marimo is trying to be a replacement for Jupyter in terms of exploratory data processing and visualization. That... was not what I used it for, so the overall experience was mostly illuminating in terms of "Boy, Python sure can be used in a lot of ways, and the resulting code can look very different."

To get a feel for marimo, I looked over some code I'd written as a Jupyter notebook that, to be clear, never needed to be a Jupyter notebook, so getting it into marimo was mostly not interesting, except for all of the typos that I found because the code was never actually run.

From the perspective of my own code, my main takeaways were that working with annotations felt not-great, and I'd have to look over the code to understand how some of this all works. Like, looking at the generated Python file, I'm not sure how the code representing the cells gets dealt with as something to actually run.

Mostly, I've come away with a greater sense of how much the purpose or tooling can change the look of the code. Like, I'd never write something that looks like a marimo notebook "by accident", and there are at least aspects of the Python that Dafny generates that I wouldn't think to write like.

If you haven't taken a look at the Python code that Dafny generates, please understand that that was a self-directed burn that does not intend to disrespect the Dafny project.

Overall, I'm very interested in the "automatic literate programming" aspect of marimo, but I'm seeing some teething problems for specific styles of code. If you're not using annotations, I doubt you'll hit any of the issues I did. (Although it occurs to me now that I think some Python code may be outright syntactically invalid to put in a marimo notebook, like star imports.)

Sadly, I don't have anything to say about the marimo module, because I... didn't try to use it. This code was really not meant for notebooks, but having it in a weird interface helps to dull the itching sensation I get when Python code is neither a fully standalone script, nor a version control repository with a full test harness set up.

Potentially, there's some better tool out there for that specific use case. "Make it easier to set up a repository with test harnesses" isn't it for me, because I am several years worth of rabbit hole into writing a task runner that accords with my sensibilities, and I haven't run across anything that obsoletes my work on MOTR. So, my best hope is probably something in the notebook space, but I'm not sure what. Maybe I just need to think more about what I'm trying to accomplish here...

Anyway, the one thing I'd really like to change so far about marimo is the tab completion bindings, but maybe that's just a matter of "it's not working how Kakoune works". Aside from that, I'm interested in trying to use it the next time I need to do actual notebook stuff.

For now, I'm going to wind down.

Good night.

Weekly Roundup 2023-11-21

2023-11-21T05:00:00-05:00

Wednesday: I filed a Mypy bug, in case the behavior in question is actually considered a bug; and tried to work around it in the meantime.
Thursday: I explained what the code involved is doing, why that requires a plugin, how the plugin is currently breaking, and how I'm trying to get insight into all of this.
Friday: I got a little distracted with some runtime and plugin improvements that are out of scope.
Saturday: I zoned out all day.
Sunday: I kept on considering stuff for the plugin that makes sense from the perspective of "generally usable library", but is out-of-scope for "I didn't want to keep on writing virtually the same wrapper code".
Monday: We traveled, and I didn't get much done.

Next week, I'll figure out what I want to work on next, since I managed to update the plugin after the last entry, but that just exposed a non-plugin-related bug. I'm thinking either back to music, some strange fanfiction I thought of, or trying to do planning work for some of the project ideas I have kicking around.

Diary 2023-11-20

2023-11-20T05:00:00-05:00

We had travel today, but I did get to think some about the plugin for MOTR.

I just threw together a few more typing tests, and honestly they just raised more questions, like, was all of this code necessary in the first place?

Hopefully, I'll have answers in a few days. For now, I should wrap up.

Good night.

Coding 2023-11-19

2023-11-19T05:00:00-05:00

Okay, so. I ended up taking things easy today, but I did put together a list of things to investigate for the dependent mapping plugin in MOTR. Basically, I need to consider:

Which methods the plugin works with, and the types of their arguments
How the methods are to be invoked
Whether a given type is to be a single concrete type, or something more complex.

There is a lot of stuff to be written to get all of this testable. Some weird combinatorial explosion, considering that I'm working with parametric types that I then want to see working sensibly in unions and the like.

Anyway, it's late and I need to sleep a lot.

Good night.

Diary 2023-11-18

2023-11-18T05:00:00-05:00

I had no focus today, so there's nothing to write about, and I want to finish this up quickly. I'll see what I manage to do tomorrow, but absolutely nothing else is getting done right now.

Good night.

Coding 2023-11-17

2023-11-17T05:00:00-05:00

Well, I got distracted, but I do have a few notes.

Currently, calling a method from the DependentMapping protocol does not (I assume) work at run time.
But I can make it work.
So it's mainly a question of what it would take to make it work in the plugin.

What exactly does "making it work" entail?

Um...

def __add__(self, other):
    return self + other

:) (glitchy)

MmmMMmmmm, that's the good stuff.

Getting that in would require some minor investment in terms of testing because suddenly the method bodies would need to be executed.

Anyway, I need to get back into the guts of this code to figure out how to change it properly. Something that just occurred to me is that I can define a "notes" variable inside my debug hook, breakpoint after it, and then I'll be able to get to the breakpoint by running Mypy, at which point I can remind myself what I was just doing by printing the notes variable, so it's all right there in front of me.

Anyway, I let it get late, so I'll just wrap up for now and see what I can do with this in the coming days.

Good night.

Coding 2023-11-16

2023-11-16T05:00:00-05:00

So, like I said, I filed the bug report, and then realized "Wait, I bet I can handle this." So here's the general idea: I've written a plugin for handling the concept of a dependent mapping, a mapping where the type of a value depends on the type of the corresponding key. I'm allowing for arbitrary relationships, which should require a concept along the lines of higher-kinded types, which Python's typing ecosystem mostly does not have. Now, a higher-kinded type is basically like, you have a type-level function between types, and you can write type-level functions that operate on type-level functions instead of plain types. So, you could have list[T] or T | None, and you'd like to be able to pass them around. This turns out to not really work; you can't pass around "a generic type with a hole in it".

However, we can instead take a type-level function, and turn it into the type of a function that transforms types the way we'd like it to.

What?

Let's turn those examples above into what I'm talking about.

class ListOf(Protocol):
    def __call__(self, __k: T) -> list[T]:
        ...

class Maybe(Protocol):
    def __call__(self, __k: T) -> T | None:
        ...

Now, these callable protocols are entirely concrete, of course; just look at the class lines.

Now, these callable protocols are entirely generic, of course; just look at the def lines.

There's no way Mypy is cool with this.

Absolutely not; why do you think we're in this mess?

We?

Anyway, I could imagine various uses for this idiom; actually implementing these protocols could be useful for some kind of data processing pipeline, potentially. But I've instead been using them as the type arguments of a generic protocol, which is like Mapping, but is not Mapping. Most of the methods on the protocol actually typecheck perfectly fine, and to bridge the gap, I had to write a plugin that transforms a select few signature and method accesses, pulling up variables hidden inside the protocols.

This worked perfectly fine for a few versions, but with Mypy 1.7.0, some new type inference code became the default, and it gives those methods much less helpful default signatures. At first I thought "Well, I need the signatures to go back the way they were", but then I realized that most of the information is still intact, just not where I want it to be, and there is a finite set of methods to update, so I can hardcode rather a lot if I need to.

Let's have a look at what I've put together so far. Now, none of this is functional code changes; it doesn't really work to rush into writing a Mypy plugin; getting small details wrong can crash it outright. So, I'm currently focused on diagnostics. Print-based diagnostics. Just stick a print statement in the plugin code, and it shows up in the command-line output. EZ.

Here's what I've got currently:

CALLABLE_SLOTS = (
    "arg_types",
    "arg_kinds",
    "arg_names",
    "min_args",
    "ret_type",
    "name",
    "definition",
    "variables",
    "is_ellipsis_args",
    "implicit",
    "special_sig",
    "from_type_type",
    "bound_args",
    "def_extras",
    "type_guard",
    "from_concatenate",
    "imprecise_arg_kinds",
    "unpack_kwargs",
)


def pprint_callable_type(callable_type, when):
    print(when, "(")
    for slot in CALLABLE_SLOTS:
        print(slot, "=", getattr(callable_type, slot))
    print(")")


def debug_hook(hook):
    def new_hook(ctx):
        self_arg = ctx.type.args[0]
        print(self_arg.type["__call__"].type)
        before = ctx.default_signature
        pprint_callable_type(before, "before")
        after = hook(ctx)
        pprint_callable_type(after, "after")
        return after

    return new_hook

Most of the hooks in the plugin focus on rewriting the method signature, so I've written a diagnostic decorator that details the information relevant to the plugin's functionality, and how the hook transforms the information that it's supposed to transform. Because it's a decorator, I can put it on a hook, and take it off, without worrying about messing up the hook's body.

I've put together a small synthetic test file, and am comparing the debug output with and without the new type inference. So far, I'm seeing that the output is mostly the same; it's just that arg_types and ret_type have Never instead of anything useful.

And... I just discovered a crash that was latent in the plugin code all along. I'm going to have to figure out how to convert that into a proper call, or at least a reasonable error telling people "Hey, you can't use the plugin like that. Stop it."

Anyway, it's going to take some more work and a bunch of notes to put together a plan for how to handle this stuff. I'd better put this entry down and see what I'm up for the rest of the night.

Good night.

Diary 2023-11-15

2023-11-15T05:00:00-05:00

I just filed a bug that I'm not confident will get fixed—

"fixed"

But I figured I'd at least ask.

That said, now that I've thought about it some more, there may be an alternative on my end to changing the behavior. If I can reach inside...

Hm...

Yes, it does appear that nearly all of the information I need is still available, and I "just" need to hardcode the remainder. I think this will end up being "switch to the old type inference, get information about what the signature 'should' look like, and add that information in".

We'll see what that ends up being like later. For now, I need to get to bed.

Good night.

Weekly Roundup 2023-11-14

2023-11-14T05:00:00-05:00

Wednesday: I made and planned some improvements to MOTR.
Thursday: I ran into some complications, and tried to work out the best way to deal with them.
Friday: I didn't really make concrete progress, but I did keep thinking.
Saturday: Eh, rough day apparently.
Sunday: I discovered bugs in Mypy, both comprehensible and incomprehensible.
Monday: I finished updating MOTR, ignoring the Mypy issues for now. It was a very easy update once the code was in place.

Next week, I'm going to try to psych myself up to file one of those Mypy bugs. And probably other things, I don't know.

Coding 2023-11-13

2023-11-13T05:00:00-05:00

I've got MOTR's code cleaned up in the rewrite branch. Now, I could get back in there trying to continue the updates or tweak the names I came up with. (Those names are overall better than what came before, but not great.) But I think I'd like to step back and find other stuff to focus on, at least until we get a new Mypy release or two.

It's probably best for me to work on getting thoughts written down for music, but I felt like trying to learn a bit about Dafny today, so I did. There are two categories of thing I'm interested in with Dafny: what kind of code does it make sense to write in Dafny, and what does it take to integrate that code with other code bases? Looking at the output from the Python builder, I'm fairly certain this code won't cleanly slot into my usual Python workflows, though there are a few niche applications where there could be potential. But in general, it seems like it'd make more sense to have Dafny target faster code and then FFI into it. I don't know how hard that actually is, though.

As far as what to write, I think I need to learn more than the basics of the data model.

Anyway, I took too long with this post, so I'm going to cut it off abruptly.

Good night.

Coding 2023-11-12

2023-11-12T05:00:00-05:00

Things that have happened with MOTR:

The Mypy 1.7 update broke things; fortunately somebody else already filed a bug.
I'm deep into the rewrite; it'll be interesting to see how badly the tests go, once the code looks reasonable.
I ran into an absolutely inexplicable Mypy bug that I'll try to get a reproducer for once everything else is cleaned up.

Fortunately, I'm doing most of this on a branch. I had a sense that the update could be hairy, and it has been so, so much worse than I was anticipating.

Anyway, it's late and I'm tired. We'll see how this ends up tomorrow.

Good night.

Diary 2023-11-11

2023-11-11T05:00:00-05:00

I poked a little at MOTR, but not enough to write about. I think I'm going to need to put it down in favor of something else for a while.

I've got some things I can mess with, but I think I mostly need to find some way to unwind. We'll see how that goes. I'm going to start by not trying very hard with this post.

Good night.

Coding 2023-11-10

2023-11-10T05:00:00-05:00

Still thinking. Need to tease out the concepts. Invocation to represent launching a command; add implicit artifacts to represent dependencies; add installers to convert executables to dependencies...

I still need to think about it, yeah.

Good night.

Coding 2023-11-09

2023-11-09T05:00:00-05:00

Hm. It turns out that things aren't quite as easy as switching to Invocation, because there are various pieces of data that have to get passed around and worked with. Like, however you create a Cmd task, doing so requires four pieces of information:

the arguments
the environment
the implicit artifacts
the failure codes that shouldn't abort

I've got a sense for how this should all fit together with some of the high-level stuff, but I'm not confident in my gut feelings here. Those gut feelings are:

arguments + environment -> invocation; this is known
invocation + implicit -> ???; this type would convert failure codes to facts
??? + installer mapping -> ????; this type would downconvert to a ??? suitable for the above; this type (modulo layout) is currently called Metadata, but, no.

I think I need to let this percolate for a bit, and I do have some other things to focus on in the meantime, so let's do that...

Good night.

Coding 2023-11-08

2023-11-08T05:00:00-05:00

I guess my plans ended up being for just a week in the future. My plan now is to create an Invocation or InvocationParameters or something like that, a container describing the information required to dispatch a command. Next, rewrite the upper layers in terms of that, and see what falls out.

I'd hoped to get more done tonight, but I at least got the environment variable representation fully cut over, so that's nice.

For now, I'm going to chill out and read stuff.

Good night.

Weekly Roundup 2023-11-07

2023-11-07T05:00:00-05:00

Wednesday: I made some plans for how to reorganize parts of MOTR.
Thursday: I made a promising start on the NaNo.
Friday: I was tired.
Saturday: I caught back up, but still tired.
Sunday: Tired.
Monday: I ran out of runway on the NaNo, and realized there were serious issues, and, like, if I don't have a plan to address the issues, how can I charge ahead writing?

Next week, I'm going to switch to some smaller projects (MOTR is made up of many small projects, okay?). For one, I want to try to put some of those changes to MOTR into practice; I think I can do something a little cleaner than what I had in mind at first. For another, I'm looking into messing around with music composition again.

Diary 2023-11-06

2023-11-06T05:00:00-05:00

Okay, so, NaNo. I'm not definitely giving up on it, but I'm not going to finish it the way I'm going currently. There are a few problems here:

I have painfully low energy and just want to lie down forever.
I've more-or-less exhausted the stuff I pre-planned.
I'm not sure if the stuff I want to write about merits something novella-length on its own.
I'm trying to show off some fancy worldbuilding I came up with, but every character currently alive is looking at it and going "So, it's a bad idea to engage with that". At the same time, if a character did do the irresponsible thing, it would probably render the narrative too confusing.

Ultimately, what I really wanted to get out of this was a test of some insultingly simple command-line tools I put together for writing, and the ultimate verdict on those so far is a resounding "maybe". Like, for sure, they don't get in the way in the way that any minimally functional editor does, but I guess I can't really evaluate how well they worked without trying to put the output through a bunch of editing passes.

Basically, if I can address all of the points above in the next few days, I'll get back into it, and otherwise I'll let it be. (Not like I wasn't already sneaking in work on other things anyway. Maybe this whole "work on just one thing for a month" thing isn't for me any more...)

Anyway, for now, I'm going to print some stuff out so I can get off my laptop and try to take things easy.

Good night.

Diary 2023-11-05

2023-11-05T04:00:00-05:00

I hit par today, and after that, I'm trying to take things easy, because I'm dealing with a lot of stress.

At least I'm relaxing.

Good night.

Diary 2023-11-04

2023-11-04T04:00:00-04:00

I'm doing a little better now, I guess. As far as writing, I'm staying on par. I wanted to try to get ahead, but it's late enough that I'm not going to try to force it. I'm a little worried I've written myself into a corner, so I'm going to hope that sleeping gives me some inspiration.

As for the rest of tonight... There is nothing else.

Good night.

Diary 2023-11-03

2023-11-03T04:00:00-04:00

So. I was really tired today, and I didn't keep pace, but at least I'm over par. I should have a few days to properly divide between rest and writing, and we'll see where that leaves me.

Right now, though, I'm just going to read.

Good night.

Diary 2023-11-02

2023-11-02T04:00:00-04:00

Good start out the gate with NaNoWriMo today. Nothing too flashy, just a solid batch of words over the course of a few hours. I'm... not totally certain what I'm going to fill the overall wordcount with, but I'm going to try not to worry about that.

I'm really tired, but that's just my system being all out of sorts from whatever messed-up stomach bug I picked up somewhere. All the same, I want to get off my laptop for now.

Good night.

Coding 2023-11-01

2023-11-01T04:00:00-04:00

I'm going to switch gears tomorrow, but until then, I've been having a look at the parts of MOTR that I want to update, and one thing that I have to deal with is that I'm not sure what my endgame was seven or eight months ago. My goal currently is to put together a tower of types, based on constructing types like GeneralizedType = SpecializedType[SomeType | T]. It looks like my thought process at the time was "So, I want to devise a more specialized type for environment variables, because the higher layers in the stack need that, and it [apparently] makes sense to plumb that back down to the lowest layer where it's relevant; oh, but I should make sure to avoid breaking backwards compatibility". Frankly, it looks like I was wrong about that last bit, because my current motrfile doesn't seem to set any environment variables. That means I should be able to mess with it however I want.

And what I'm currently thinking is:

Ditch the LegacyEnvVar concept, and get everything working with EnvVar.
Create a compound type that works with arguments and environment variables via a single type variable.
Create another compound type that bundles that type with the implicit io, using a separate type variable. (Might not go with this, not sure.)
Make a new helper function to take these new types.
Propagate these types up the layers of abstraction.
(NOTE: I cannot change the Cmd type currently, because it's being used directly by the Motrfile.)
This should get rid of a Metadata type, at the cost of potentially adding more fields to another class that needs a redesign anyway.

I won't try to work on this tonight; at some point I'll be back on this. For now, I should wrap up, and figure out how sick I am or am not...

Good night.

Weekly Roundup 2023-10-31

2023-10-31T04:00:00-04:00

Wednesday: I was sick a week ago. Guess what's happening now!
Thursday: I pondered some more renames for MOTR.
Friday: I made some progress on planning for MOTR, but didn't implement the plans just yet.
Saturday: There was enough of a plan, so I put it into effect, then made some other improvements.
Sunday: I realized that some of the code in MOTR is probably needlessly strict, so I need to look into what contracts actually need to be upheld.
Monday: I finally remembered what command.Metadata is meant to accomplish. Such is the obfuscatory power of a really, really bad name.

Next week, I'm going to try to spend a little time planning out the changes needed to remove command.Metadata. However, after tomorrow (keeping in mind the weird dating system for these blog posts), I'm going to have another place I'm going to be trying to put my focus...

Coding 2023-10-30

2023-10-30T04:00:00-04:00

Okay. I've got some more insight on one of the Metadata classes, but it hasn't translated into a new name yet.

Here's the deal:

A command, like you'd dispatch on the shell, has some arguments, and environment variables associated with the execution. It also may rely on or cause side effects that are not specifically mentioned in the arguments or environment variables. This is expressed within the low-level interfaces exposed in current releases of MOTR.

A concept at a higher level is an Executable, which is approximately "anything that requires some kind of installation process that MOTR is to handle, and is capable of processing arguments, environment variables, etc" This could be either a shell script, or the slightly less obvious "argument to python -m"

The Metadata class sort of combines these, by having containers corresponding to the lower-level ideas, but allowing Executables in addition to the normal data types. It then provides facilities to convert these Executables down to the normal data types.

So, the problem is, obviously Metadata is a bad name for this, and I should come up with something better. But I'm not sure what yet. One possibility is to push some of the concepts to the lower level, which would effectively remove the class, via converting its functionality into a single function. Ultimately this wouldn't get rid of the problem of naming a class, but it would simplify it, by making the class to be named smaller and more focused.

I think that'll get things in a good position, though it'll be a rough merge up to the future topic. For now, I'll be glad I've come up with a plan.

Good night.

Coding 2023-10-29

2023-10-29T04:00:00-04:00

All right, let's see. More coverage for MOTR, and now I'm staring at some of the remaining lines, and thinking "Hm, it seems like some of this validation is too strict." That's going to take some careful thought to be sure of one way or the other.

Like, the matrix.directory decorator combines together some amount of SelectionDirectory and Matrix objects, along with a function to map the values all together. The question I need to consider is, what can go wrong when it's guaranteed or possible that the matrix will allow iteration over labels outside of the directories? To address that, I'm going to need to once again trace out the code that can potentially be hit by the highest levels of abstraction in this system. For now, I'm not up for doing that work; maybe tomorrow.

Good night.

Coding 2023-10-28

2023-10-28T04:00:00-04:00

Did the simplest possible layout that I'm comfortable with; the result is a little iffy currently, and I might rework it slightly later, but it's fine for now.

All of that cleanup got me comfortable writing more tests for the matrix module. Just a few more tests, and it should be covered, and I can start working on other modules.

For now, though, I'm just going to take things easy.

Good night.

Coding 2023-10-27

2023-10-27T04:00:00-04:00

I did a little research, and I've decided to go with DependentMapping. I'm still trying to figure out how to organize things. Because, I want to bring in sample magmas that other code can drop into update_with() calls.

I think I want this to look something like

import dependent_mapping.magmas

...

CustomMapping = dependent_mapping.DependentMapping[_CustomCallableProtocol]

That's straightforward enough to describe, but I've lost track of the guidance on what's acceptable to put in __init__.py files, and the DependentMapping protocol is... beefy. (That said, some of the opinions I remember seeing about __init__.py came from people who also said that slash args were useless, which is a fascinating position to take with respect to a project I was working on at the time, which contained some functions that needed to take a limited number of positional arguments, and totally arbitrary keyword arguments.)

So, I'm going to publish this entry, and start reading up on what is and isn't acceptable in __init__.py files. And get to bed at a somewhat reasonable time.

Good night.

Coding 2023-10-26

2023-10-26T04:00:00-04:00

I'm feeling better now. More focused. Let's not squander it.

I tried out those renames I was thinking of for MOTR, and I think that they did improve things. To be sure, I'm going to need to step away from the code for a bit, then read over the modules I changed to see if they're any clearer.

After getting those changes in, I find myself focusing on a helper function that I wrote into Axes.combine. That function clearly needs to go together with the whole... "functions only defined on finite sets" concept that ties a lot of the new layers of MOTR together. To me, that means that the relevant module needs to be moved higher in the tree, next to the saltate module. And that suggests that I should try to come up with a non-awkward name for it.

per_item_parametric_mapping to... Hm. Basically, the idea is that this type represents the same concept as a parametrically typed function, but it's only defined for values that are specifically set.

... What's this paper?

... All right.

"But, Max. Mypy doesn't let you define a type that acts like that."

Not without convincing it doesn't.

Anyway, what to call this... After the renames I've done, shortening it to ParametricMapping doesn't sound unreasonable to me. Need to figure out the right naming conventions around the magmas, though. The annoying thing about ParamatricMapping, when I think about it, is that you need to look at it and think "Well, Mapping is already parametric, so how is this different?"

Well, I'll take some time to think about this, and see what I feel like doing.

Good night.

Diary 2023-10-25

2023-10-25T04:00:00-04:00

I was too tired to focus today, and there's no way I can improve the situation by trying to force something now. For what it's worth, I did pile in a few more project ideas, so things promise to get interesting in a few months.

Hopefully I'll have more energy tomorrow.

Good night.

Weekly Roundup 2023-10-24

2023-10-24T04:00:00-04:00

Wednesday: I thought about names in MOTR, and wasn't sure if I wanted to go the finally-obvious route.
Thursday: I still wasn't sure about the names, but I was able to write some code that will eventually be useful, I think.
Friday: I came up with absurdly many projects to work on... at some point.
Saturday: "Well, let's get back into MOTR for now... but also attempt NaNoWriMo."
Sunday: Settling on the obvious name gave me the motivation I needed to write tests again.
Monday: I got more test coverage... but ran back into having problems with the names.

Next week, I'm going to keep on pondering the name ideas, and see how I feel.

Coding 2023-10-23

2023-10-23T04:00:00-04:00

Hmmph. I'm pushing up the coverage on the matrix module, but it's getting to the point where I have to consider the names of things before I feel comfortable writing more tests.

See, there are a few supplemental classes in that module that I'm not feeling totally on-board with the current names of: AdaptiveMatrix and MappedSelection.

Let's consider MappedSelection first. Basically, it's a single selection label, plus a function to map from the type of the selection label to something that can be slotted into a Path. Put together, it allows for the creation of two matrices which can be freely combined because they have the same axis structure. The label itself and the matrices are relevant in different parts of the code.

One of those parts of the code is the AdaptiveMatrix, which combines a variable-length tuple of MappedSelections with a Matrix[T], which is constrained to iterate over exactly the same labels as are present in the tuple of MappedSelections. The AdaptiveMatrix is one of three possible things that can end up in a sequence of command-line arguments(ish):

static text
the paths of artifacts passed between commands
auxiliary data required by specific matrix values (AdaptiveMatrix)

This sounds niche, but everything breaks if I don't have it, so.

When I look at the code, I think this could potentially get a bunch more complicated, but maybe not...

Anyway.

The key perspectives of the MappedSelection:

It can provide a Matrix that exposes a value of an arbitrary type.
It can provide a Matrix that exposes a value suitable to use as a directory name, and these matrices are perfectly correlated.
It can provide a selection.Label that is used to constrain and determine the structure of other matrices.

I'm not going to put in the effort tonight, but now I wonder if something like SelectedDirectory and MatrixDirectory would strike me better. I'll sleep on it. For now, I've got other stuff I'll focus on for a bit to clear my head.

Good night.

Coding 2023-10-22

2023-10-22T04:00:00-04:00

Okay, so I gave myself sort of a break today, focusing on improving various bits of MOTR. I finally settled on calling it a Matrix, and this cascaded into a bunch of renames that made things clearer. And made a few other changes.

That added up to enough clarity and motivation that I finally added a line of test coverage. Now, I'm not sure how best to handle the remaining lines in the matrix module, because the vast majority of those lines are validation failures or similar. I guess there's nothing for it but to carefully construct a bunch of axes "wrong" to make sure they hit the validation errors...

To get that done, I guess I just need to be in the right frame of mind to power through the monotony. Anyway, it's past time I meant to post this; I need to get to bed now.

Good night.

Coding 2023-10-21

2023-10-21T04:00:00-04:00

The document of ideas is even longer now, so that's... a thing. I've come up with some ideas to integrate some Python libraries with the way that I, and, near as I can tell, nobody else, write tests, so that'll be really interesting. Unfortunately, I still don't have the discipline to Just Use tox, Nox, or Hatch's matrices.

With that in mind, I'm going to try to put some renames into practice in MOTR, now that I know what one of the Metadata classes should be called instead. (I just checked, and there are apparently just two? I must be using them a lot, because it feels like they're everywhere.)

With that said, however... Due to a confluence of factors, I'm interested in giving NaNoWriMo another try this year. (The factors in question are, I have finally "properly" developed an idea that I have been utterly failing to explain to people for like a decade, and I just threw together some tools for drafting that I want to see in serious action.)

So, the plan is to kind of disappear for a month. That is things working as intended.

We'll see how things actually go when the time comes. For now, sleep.

Good night.

Coding 2023-10-20

2023-10-20T04:00:00-04:00

I messed with last night's "greenfield development" a little; it should be about ready to by put into use, assuming Mypy doesn't have any weird objections to it.

For now, though, I suddenly had a bunch of ideas for stuff to work on besides MOTR, so I'm going to focus on categorizing and prioritizing dozens of things I could be looking into.

...

I managed to roughly bin everything. I haven't prioritized them yet. In terms of concrete goals, there are three-ish things I can look into:

Music theory and synthesis stuff, with an eye towards stuff besides 12EDO
Prototyping interpreters for Impliciula
Going back to my dice rollers project

We'll see if I manage to settle on one. One serious obstacle I have is that, unless someone just gives me a toxfile or a noxfile, I'll probably end up longing for MOTR, so the best bet may actually be looking into the options that specifically don't involve Python...

Anyway, I don't want to stick around too much longer right now.

Good night.

Coding 2023-10-19

2023-10-19T04:00:00-04:00

Quick name thought: some CI products call it a "matrix", but... On the one hand, I want to say "tensor" because more dimensions, but apparently a tensor is more identified with a generalization of a linear transformation, and linear transformations can be represented by matrices, but the grids the CI products generate don't have anything to do with linear transformations.

What we have is some kind of generalized Cartesian product and the ability to map over the result. I mean, except that the Cartesian-product-like-thing is idempotent, and commutative, and associative. So, like, at that point, what is it? (I guess if an ordering is defined over the labels, then we can fold the Cartesian product over an ordered set of labels mapped to typed sets?)

Anyway, I want to change gears, so I'm going to prototype some code that I won't be able to use for a while.

...

Okay, done for now, got a few things to think about while I wait to be able to use this. Basically, I wanted a wrapper around contextvars, using contextlib, so I can have context managers controlling the values of box and selection instead of passing them through every function, mostly unchanged. This is pretty easy to throw together, but I did add just enough functionality to raise questions. Like, the basic functionality is to have a get() helper method, and a set() method that returns a context manager. But I anticipate that I'll want to express the logic a lot of the time as "update whatever value there is, with this function" rather than "temporarily replace the current value with one that happens to be related to it". So I wrote an update() context manager, and it's like, if there isn't a value in the variable currently, do I want that to fail outright, for the caller to specify a default value to get(), or for the caller to provide a default value that replaces the act of calling the function passed in?

I guess I'll need to figure out what I want to happen in my actual use cases for this stuff. I think I'd want the get() to fail, and a helper function in the stack to catch the LookupError and convert it to something MOTR-specific? Which is certainly easy to implement from the standpoint of the code I'm looking at now...

Anyway, I'm tired and foggy. Time to wind down.

Good night.

Coding 2023-10-18

2023-10-18T04:00:00-04:00

I discussed naming concepts in MOTR with my wife, and I came up with A Breakthrough. Just need a couple more...

In short (because I want to rest my eyes), one of the key concepts in (the high level code in) MOTR is currently called Parametric, and has an attribute just called metadata, which is... a Metadata. After a bunch of discussion, I realized that the Metadata is roughly analagous to the concept of "shape" or "axes" of a multi-dimensional array. Which, you'd think would settle "what to rename Parametric", but...

Okay, so, Parametric corresponds to what some continuous integration systems call a "matrix", but when I searched on the web, it seemed like this usage is, in some sense, common but not standard, if that makes any sense? I was looking because I'm used to thinking of "a matrix with an arbitrary number of axes" as "a tensor" (though I admittedly don't use tensors much), but I thought to myself "If I call it a 'tensor', I bet people will ask why I didn't call it a 'matrix'".

Anyway, even if I'm not sure what I want to call it, that line of thinking did get me thinking about some potential features I could add. Basically, it wouldn't be too hard to support excluding particular combinations of axis values, though it would be easier after a planned refactor that I don't want to do while I've got the kind of coverage gaps I've had for months.

Final thought: staring at the code relevant to that feature idea is forcing me to re-ponder the tension between "Wanting the Mypy coverage report to look good" and "Heehee itertools.product(*(itertools.zip_longest(...) for x in y)) go brrrrrrrr". Like, really, that "line" of code needs an essay explaining what's going on...

Anyway, I have nothing more to say, and no coherent exit strategy, so,

Good night.

Weekly Roundup 2023-10-17

2023-10-17T04:00:00-04:00

Wednesday: I ran into Problems with MOTR, which were, on balance, mostly self-inflicted.
Thursday: I did some unpleasant work on MOTR, and did some easier stuff to take the edge off.
Friday: I finished up with the easy stuff, and then I was tired...
Saturday: "You know what would be fun? Cataloguing every global name, attribute, and function name in the code base."
Sunday: I went several layers deep trying to figure out which names need to be changed first.
Monday: I circled back around to names that have given me trouble before.

Next week, maybe I'll keep on messing with the names, or maybe I'll come up with something else to do.

Coding 2023-10-16

2023-10-16T04:00:00-04:00

Traveled back, and had time to consider the names in MOTR. I've ended up having to reconsider something that was bothering me last time: I've managed to build part of the system thoroughly and pervasively around a concept that I can't figure out a good name for. It's as if I'm a carpenter who is perfectly skilled on a technical level, but I don't know the word for "join", so I have no idea how to explain what I'm doing.

Basically, suppose you have a data type called a Label, which has a type parameter with no run-time effect, and it compares by identity instead of by structure. This means you can (provided the proper plugin code exists) construct containers that map a Label[M[T]] to a N[T], for some M and N. I call the type mapping Label[T] to T, Box. The metaphor here is that the box has a label on it, and the label indicates what's in there. Now, suppose M[T] is tuple[T, ...]. If you apply the corresponding label to a Box, then the value is a collection of T. But if you have a container with the given M, but N[T] is just T, then that same label maps to a single T. This could be a choice of T from the collection in the Box, and because it's a choice, I call it a Selection.

I went a little crazy with custom container types once I got the plugin working, but those two types end up enabling a lot. Clearly, once you have these types, and the right labels, you can convert a function that takes values of types into a function that takes a Box. And if some of the labels are to collections where you just need one value, you can get that value out of a Selection. In the context where I'm using these functions, it makes sense to want them to, instead of returning a value directly, actually return a generator that yields something called Facts, and then terminates with the value. There are some obvious questions here about whether these values can actually be expected to be in the relevant containers, and pinning down the answers to those questions is where a lot of the code at this level of abstraction comes from. But the first question I think I need to answer here is, given a function that takes a box and a selection, and yields facts before stopping with a value, what do we call that?

(There are some changes I want to make to the code later that will somewhat obscure this relationship at a syntactic level, but I'd rather ignore that caveat.)

The idea of all of this is that the top-level invocation will yield the facts associated with every possible selection that can be made from the box. (Where the definition of "every possible selection" is actually very subtle and fiddly.)

I wonder if one of those parenthesized statements could help. Like, if the underlying function is just a nullary function that returns a generator. (Or possibly just an arbitrarily resumable generator...) That object is now wholly defined by its output behavior, which is to calculate a value and yield facts as a side effect.

This feels like there's something to figure out from this perspective, but it's like I'm missing a piece...

I'll just have to think about it later, because it's getting late and I feel my mind fogging up.

Good night.

Coding 2023-10-15

2023-10-15T04:00:00-04:00

Okay, I got distracted today, and there was travel, so this is going to be a quick entry. I worked through the names in MOTR a little more, and I'm right now in the area of the Artifact protocol, along with some of the helper classes it relies on. The Artifact protocol is suffering from a lack of evocativeness, with method names like convert, forbid_narrowing, and exposed_selections.

convert just needs a more evocative name
forbid_narrowing should be better aligned with what the implementations ultimately do, which is focused around the "exclusive fields" of a Parametric.
exposed_selections points towards the parametric.MappedSelection helper class, which stands out to me as the biggest problem here.

The point of MappedSelection is that it extracts a value from a Selection container, but it also offers a way to convert that value to a segment of a string-based path. It is made via the parametric.map_over decorator and the installer.adaptor helper, which is a name that extra needs to be updated, because it hasn't gotten to the current round of naming yet...

Anyway, I think that class isn't being used to its full potential yet, which complicates things, but it is being used. It's too late right now for me to determine more. I'll try to understand this code better in the morning.

Good night.

Coding 2023-10-14

2023-10-14T04:00:00-04:00

Well. I ended up looking over some of the tests representing what a typical motrfile would look like, and I concluded that it wasn't great. I took some notes on reducing clutter and making it easier to get the code right, but what I ended up deciding the code needs, is a holistic look at all of the names, and how they fit together, or sometimes, really don't. So, I've started to catalogue every global variable, class, field, function, and parameter name in the project by module, and to include any notes I come up with on what to do with the names.

I've already come up with a few ideas for improvements, so let's see about getting those in before I write up too much more of this stuff.

For now, though, sleep.

Good night.

Coding 2023-10-13

2023-10-13T04:00:00-04:00

All right, I've made all of the obvious typing-related changes to MOTR's code. If I'm not going to write tests for plugin-related cases I can't figure out how to hit, I guess I need to go between improving test coverage and improving names.

...

I suppose one other thing I can do is work on documenting the saltate plugin thoroughly enough that I feel confident using it, and then getting test coverage up that way...

Anyway, I'll see what I'm up for later, because I'm really tired now.

Good night.

Coding 2023-10-12

2023-10-12T04:00:00-04:00

Okay. I managed to find an acceptable workaround for the Mypy 1.6 stuff, though I remain somewhat salty. I did a merge up to future just to hopefully make subsequent merges a little less bad, and now I'm staring at the missing coverage in the plugins, thinking "Do I actually care? Is this a condition that can actually be hit?"

Probably the sensible thing to do for now is to focus on documentation and deriving test cases. Then, later, I can reconsider these uncovered lines, and either figure out tests for them, or ask for help with those parts of the plugins.

...

Wait a sec. I bumped the Python version in the dev environment. That means I'm good to update more of the typing syntax. Necessary, no, but nice-to-have, yes.

I'll get on that tomorrow, because right now I need to get to bed.

Good night.

Coding 2023-10-11

2023-10-11T04:00:00-04:00

I regret to inform that getting back into MOTR is super-cursed. Things I ran into:

It only works on Python 3.11 because of reasons.
I somehow managed to write the motrfile against a version that I never published, so I had to monkeypatch the internals at the beginning of the motrfile.
Mypy 1.6 has a behavior change that I want to call a regression. Basically, it breaks using overload and singledispatchmethod together, under some circumstances.
The saltate plugin had a bug, and I never noticed until I happened to throw together a test case that demonstrated it. Unpleasantly, the bug was in one of the cases that aren't even a little hexed on a conceptual level, so I don't even have the excuse that the relevant behaviors are confusing; the code was just wrong, and it may still be wrong...

I need to wrap up, because I'm having a bad time right now, and if I stay up any longer, I'll just make it worse.

Good night.

Weekly Roundup 2023-10-10

2023-10-10T04:00:00-04:00

Wednesday: I got enough help to be comfortable switching to Kakoune.
Thursday: That said, there is still something wrong with my custom rc file. However, I decided to think about other things.
Friday: I pondered some more how to streamline my Python setup.
Saturday: I got enough pondered to be ready for the switchover, more or less.
Sunday: The switchover went off without a hitch. Neat.
Monday: Now that my Python setup isn't, like, kind of unreasonable, I considered trying to get back into Python development, which got me considering the elephant in the room...

Next week, I'm going to be reviewing the code for MOTR again. According to my memories, if I can get the saltate plugin to be reasonable, then I shouldn't have any major trouble understanding and testing the rest of the new code. So, it goes:

Review the generic attrs code in Mypy, and check for any changes since I last touched the saltate plugin.
Update the saltate plugin and associated tests.
Once I have that working and covered, merge up to the documentation branch, study the diffs, and document anything that is confusing to me.
Use that knowledge to plan new tests.
Try to get to a point where I can finally cut new releases again.

Coding 2023-10-09

2023-10-09T04:00:00-04:00

I futzed around with tooling a bunch today. I think I need to get back to writing in order to try this software out.

One thing I'm going to need to consider, as I look over this and think about doing Python development again, is whether I'm going to be comfortable with an existing task runner, or if I'm going to end up trying to revive MOTR, which may need some help to get running with the new stack.

Oh well, I tried to come up with much more to say, and I didn't. We'll see how I feel about all of this later.

Good night.

Coding 2023-10-08

2023-10-08T04:00:00-04:00

Okay, if you're reading this, that means that switching my Python stack worked. I believe I verified this earlier today, but one more test can't hurt. I'm excited to see how my new systems work out, because I don't yet have much hands-on experience with them. I'd like to say more about this, but it's late, and I don't know what else I'd say right now.

Let's see what I think of tomorrow.

Good night.

Coding 2023-10-07

2023-10-07T04:00:00-04:00

I've put together all of the pseudocode I think I'll need to close the gaps in functionality I'll feel from switching off of pyenv. I've looked up most of the information I'll need to translate that pseudocode into fish shell functions.

I should be good to carry out the switchover tomorrow, but I'd better do it early, because the code supporting this blog is easily the most convoluted environment I'm going to want to have up and running.

I'm trying to think of whether there's anything else from today that I'd lke to talk about now, and nothing comes to mind. I'm going to go read.

Good night.

Coding 2023-10-06

2023-10-06T04:00:00-04:00

Okay, I kind of gave myself a rough time today, so I'm going to try to go a little easier on myself right now.

I mentioned yesterday wanting to rework my Python setup. My goal is to put together a better experience than I have currently. Right now, my Python installs are provided by pyenv, and, through a mix of existing and hand-rolled plugins, I manually-ish set up virtual environments for running scripts and having the requirements for projects.

Problems with this include: too many Pythons, and, relatedly-ish, something is always outdated.

One aspect that I don't have a plan for yet is that I can set a virtual environment per directory. (I do have an idea, which is "copy the stuff I'm doing with opam" ("and also try not to think too hard about how my OCaml tooling is set up in NeoVim but not Kakoune"))

Aside from that, though, here's the idea:

Stop activating pyenv in my config.fish file, but leave the entire directory tree in place
Get Python from deadsnakes PPA
Punt on installing PyPy because I don't use it for much
Configure pip to require an activated virtualenv
Create virtualenv for pipx and install it, set up shell integration
Install a bunch of binaries
Figure out the logic for directory-based virtualenvs, and do it

All of that kind of has to be in one day so I can actually publish my post when I'm done, so I'm going to try to do that last step tomorrow, then pull the trigger on all of this over the weekend. For now, I'm going to wind down.

Good night.

Coding 2023-10-05

2023-10-05T04:00:00-04:00

I now understand Kakoune a little better, but I'm still doing something wrong, but, like, just exactly wrong enough that it seems to work perfectly when I paste the commands in, but fail when I run the command that eventually invokes them. I haven't made sense of that yet; I'll formulate a question for IRC eventually.

Aside from that, I'm pondering reworking my Python setup; I have some ideas, but I need to do more research and go over exactly what my requirements are. Regardless, I don't have it in me to write anything else now.

Good night.

Coding 2023-10-04

2023-10-04T04:00:00-04:00

Got some help from IRC, and now I'm blogging in Kakoune. There are a few things I have yet to work out:

What is the workflow for using next-hunk and prev-hunk? I don't have any evidence that my code isn't translated properly, but the first issue to address is that I still don't know how to use the Git version.
I broke annotate (blame). I'm not sure how, but I did. For extra infuriation points, this is code that I specifically tested manually because it looked so gnarly, and it worked fine then!
There's probably other messed-up stuff in there that I don't know about.

That said, it makes more sense to me to focus on getting more experience with Kakoune, and I can come back and try to fix the rc file later.

Aside from that, I'm going to try to wind down a little early tonight, because of scheduling nonsense.

Good night.

Weekly Roundup 2023-10-03

2023-10-03T04:00:00-04:00

Wednesday: I tried to sketch out Impliciula's AST based on vibes, and learned that sketching out an AST based on vibes is probably a bad idea.
Thursday: I discovered an OCaml feature that I hadn't suspected the existence of (in part because I'm not reading the documentation in any kind of order), and had to go lie down.
Friday: I figured out some of the things to be done with that feature, meaning that the presentational barriers to implementing it are probably down, so I just need to deal with the technical aspects...
Saturday: I started looking into what it would take to make pre-commit work with Mercurial along with Git. I would take a lot.
Sunday: I redid my towncrier fork for the heck of it.
Monday: A little more prep work for pre-commit, that has me thinking now that whatever cursed shims are required should probably exist in their own package, to avoid saddling the project with tech debt.

Next week, I've got some ideas for what to work on, but I'm not totally sure yet.

Coding 2023-10-02

2023-10-02T04:00:00-04:00

I would like my laptop to stop crapping out. Not in the sense that it no longer turns on at all, mind you...

Anyway.

I finished the towncrier changes, but they're not tested, so what happens on usage is a surprise. I've got it handy to mess with later, but I didn't actually have anything I want to use it with currently.

I went over the different hook types used by the pre-commit tool, and tried to match them up with the hooks provided by Mercurial. The ways to do so are not always obvious...

Anyway, unless I find some guidance on Kakoune, I'm going to have to find something else to do next week. I'll think about that some after I post this. Nothing more to talk about tonight, though.

Good night.

Coding 2023-10-01

2023-10-01T04:00:00-04:00

All right. Quick post because I got distracted.

I went through the syntax highlight in my hg.kak file, and hopefully got it all properly updated. Now all that remains to clear up is when I can expect to usefully use next-hunk and prev-hunk.

Failing that, since my account is in limbo, I took a look at towncrier.

It turns out my branch is so hopelessly out-of-date that it made more sense to make my modifications completely from scratch. I've just gotten started with that, but I believe what I end up with is going to make more sense than my original modifications. In part because I looked up a few more of Mercurial's help files, and in part because some code changes mean that my changes slot in a little more nicely, at least for now.

I'll try and get it all put together tomorrow, because it should hopefully be pretty quick, now that I understand the basics of what's changed.

Anyway, it's late and I should get to bed.

Good night.

Coding 2023-09-30

2023-09-30T04:00:00-04:00

Sometimes I have simple desires with complicated realizations. Like, what does it take for me to be able to get away with using Mercurial as a GitHub client? One thing I want is Mercurial support in towncrier, which I have in a fork that is probably woefully out-of-date.

Somewhat more generally used, however, is pre-commit. If I could install the pre-commit hooks in a clone made with hg-git, that'd be the dream. Looking at the way the internal git module is used within the pre-commit repository...

main.py: git.get_root, git.check_for_cygwin_mismatch
story.py: git.init_repo, no_git_env
staged_files_only.py: get.intent_to_add_files
commands/run.py: git.get_changed_files, git.get_all_files, git.is_in_merge_conflict, git.get_conflicted_files, git.get_staged_files
commands/autoupdate.py: git.init_repo, git.get_best_candidate_tag
commands/try_repo.py: git.head_rev, git.git_path, git.get_staged_files, git.commit
meta_hooks/check_hooks_apply.py: git.get_all_files
meta_hooks/check_useless_excludes.py: git.get_all_files
commands/install_uninstall.py: git.get_git_common_dir, git.has_core_hookpaths_set

But also a bunch of places that use git inline...

In addition, there are the questions of how to translate between "hooks are script files" and "hooks are configuration entries", and between different hook names.

Unfortunately, it's late now. I'll see what I can get planned to do tomorrow.

Good night.

Coding 2023-09-29

2023-09-29T04:00:00-04:00

Okay. Polymorphic variants. I've come up with an expression syntax that I think is acceptable, and that I don't have anything else I'd want to use it for, so that's part of that language feature worked out: if I want it, it's possible to write the expressions concisely.

Now let's look at the types. My whole "symbol to English translation" schtick means that | becomes or as usual; furthermore, & seems like it should become and, and > becomes open and < becomes closed, maybe. At least as a first draft.

I would like to get some hands-on experience with these ideas, but that's going to have to wait, because my energy levels just crashed.

Good night.

Coding 2023-09-28

2023-09-28T04:00:00-04:00

My phone and my NAS conspired to delete a file I was working on, and the latest backup was missing stuff from yesterday, so I had to rewrite that section. A little annoying, but whatever.

Anyway, let's take a look at how to put together a syntax... Transcribing stuff from the OCaml reference, and... Hold up... What's a "polymorphic variant" for?

...

OCaml has type unions.

...

Why am I surprised?

Okay, before I do anything else here, I need to work out what I think this would look like in Impliciula.

...

It seems to me like I can either keep the lack of explicit declarations, but require a new sigil (and I'm now thinking about whether it's possible to avoid the sigil that I'm already using), or avoid a new sigil, but need some kind of special type declaration form.

I'm going to need to think about this more, to see if I want to do anything besides "not implement this". But for now, I'm tired.

Good night.

Coding 2023-09-27

2023-09-27T04:00:00-04:00

So, getting answers about Kakoune is presumably going to happen at some point. I could try to work out the syntax highlighting, but I ended up thinking about Impliciula.

What I think makes sense for Impliciula is to translate my various ideas so far into a specification of the abstract syntax tree. This has the advantage of being mostly isolated from the gambles I'm taking with the surface syntax. This should end up being a relatively streamlined expression syntax, with some somewhat baroque statement syntax.

...

I was trying to kind of wing it and put together something equivalent to (a subset of) what OCaml can express from memory and vibes, and having tried that, I would recommend... not doing that. When I have the energy and focus handy, I need to basically try to straight translate bits of the syntax specifications for OCaml and Koka.

Once I have the abstract syntax types, then I can experiment with developing a basic interpreter, to answer questions such as "can I actually give Impliciula a well-defined semantics?" For now, though, I need to rest my eyes and get to bed.

Good night.

Weekly Roundup 2023-09-26

2023-09-26T04:00:00-04:00

Wednesday: I wrote some standard library code in Impliciula, and considered how it was coming out.
Thursday: I got deep into a rabbit hole around naming and implementing "fold" operations.
Friday: I developed more questions about how to do things in Impliciula, and started looking into other text editors, because at this point, I only use Neovim to write this blog.
Saturday: I finally realized that multimethods and gradual typing are a tricky combination. I also looked into Kakoune a bit.
Sunday: I thought more about Impliciula and Kakoune.
Monday: I replaced an AWK script in Kakoune with a shell invocation of Mercurial which constructs a Kakoune scripting command using a template. It was hard to face the world after that.

Next week, I'm going to try to wrap up the Mercurial helper for Kakoune, and switch to using Kakoune for blogging, and try to switch back to it for things that I ended up using Kate for somehow.

Coding 2023-09-25

2023-09-25T04:00:00-04:00

What remains to be done for the Mercurial helper for Kakoune:

Figure out the differences between Git and Mercurial relevant to the syntax highlighting definitions.
Figure out how jump_hunk() is supposed to, like, do anything. Like, when do I want to be using the sub-commands that invoke this function?

Other than that, it's overall been a pretty straightforward update with nothing too crazy.

A-hem

Come on, they don't need to see—

Paste the line, Max.

... evaluate-commands %sh{ hg annotate "$@" -T "set-option global hg_annotate_flags %val\{timestamp}{lines % ' \'{lineno}|{node|short} {date|isodatesec} {user|person}\''}"}

Yay!

The eagle-eyed among you will notice that this should be slower than the code it's replacing, since I couldn't find an equivalent to the --incremental flag. Eh, at least the code is shorter.

Anyway, quick Impliciula thought: I'm doubting that I can make "gradual typing in the sense of a particular shape of the type system" work, but maybe I can do "gradual typing in the sense of carrying out local type inference". This is a compromise in terms of library design, but it should still allow tossing off short scripts without thinking too hard.

Anyway, I'm tired, and I'd like a good night's sleep without the vaccine side effects, so let's wrap this up...

Good night.

Coding 2023-09-24

2023-09-24T04:00:00-04:00

Quick thought on the problems I was pondering about Impliciula: possibly, values could be tagged with a type that includes all of the relevant information. It's not obvious to me how this would get filled in under some circumstances, though. I'm going to need to find papers on this kind of thing...

Anyway, I spent more effort today on my attempt to convert Kakoune's Git helper to Mercurial. Currently, I'm trying to figure out what the AWK function in the blame handler does, and to what extent I can just, like, ignore it. Like, it kind of looks like I should be able to construct the line-specs data directly using a properly laid-out template? Hopefully I get that figured out in the next few days. For the moment, I have to once again try to sleep off the vaccine side effects.

Good night.

Coding 2023-09-23

2023-09-23T04:00:00-04:00

As I was thinking about other aspects of Impliciula, I realized that I've been glossing over an issue that has foundational implications: various features that I want depend on the exact type of expressions, which means that the resolution associated with those features has to happen before any type erasure. I'm not sure what options I have for satisfying that kind of design constraint in a dynamic context.

My basic suspicion is that I outright cannot support dynamic multimethod resolution without something really clever to keep the type names under control.

Anyway, I'll keep thinking about whether there's some way around that, but aside from that, I've been looking at Kakoune's built-in Git scripting, and trying to convert it to Mercurial scripting. Frustratingly, this involves getting a better understanding of Git, just so I know what some of the commands even are.

Frankly, with the vaccine side effects doing their thing, I'm a little amazed that I got as far as I did, and I'm going to wind down for now.

Good night.

Coding 2023-09-22

2023-09-22T04:00:00-04:00

Minor meta note: I somehow accidentally stopped using Neovim for most purposes. So, while I'm in that situation, I figured I might as well look into alternatives. I've been hearing stuff about Kakoune and Helix, and because I literally only use Git in depth when I'm being paid for it, I decided to look into Kakoune since it should in theory be more amenable to getting it working with Mercurial and maybe Pijul. But doing that means I immediately need to understand how to write a .kak file, so that's... a thing.

Anyway, I was thinking some more about the whole catamorphism rabbit hole I went down yesterday, and I realized something that I need to deal with... Impliciula currently inherits Koka's usage of curly braces for blocks/anonymous functions, so it's not obvious to me how to make OCaml-style records work, but the interesting syntax I'm trying to make work means that I can't have Koka-style structs. I have an inkling of an idea for how to make this work nicely, but it needs some time to stew. I'll at least try to explain it for now...

I'm right now working under the assumption that blocks can have different internal semantics, and even syntax, in different contexts, where context is indicated by a prefix. So, for example, a signature block can contain a bare type declaration, while a structure block has to assign a type expression. As such, the easy solution is to come up with "record" and "record type" blocks, and see if I can come up with better names.

I don't have anything else in me for tonight, so let's leave it there and hope I'm vaguely rested tomorrow.

Good night.

Coding 2023-09-21

2023-09-21T04:00:00-04:00

I tried to come up with a nice way to deal with some of the OCaml standard library code that I was having trouble translating to Impliciula, and I think I just made more problems for myself.

Here's the deal: various standard library types have a fold function defined which, in the cases I've looked at, takes named arguments that are named after the different constructors for the type, and those arguments are functions that convert the relevant variants to a return value. I was looking at this like "I do not want to figure out what kind of noun phrase would accommodate those parameters". "... But what if, all of those named arguments were packed into a record type?" At that point, the record type is *checks the giant pile of open web browser tabs* ... isomorphic to the evaluation function in an F-algebra? I think?

Now, the normal way to define an F-algebra is to put together an endofunctor that, I think, represents the partially applied computation. The equivalent with a record type for the evaluation function is to represent the partial evaluations as parameters of each function in the record. So each function in the record would be receiving a destructured variant and the intermediate calculation, and those can be cut down based on the variant...

I'm going to need to try sketching implementations both ways, because I can't figure out whether I prefer to be closer to how the OCaml functions sort of look, or to the theoretical description. Like, is it better for all implementors to put together a when-is-be block, or to construct it in one place per type using raw materials from everywhere? The former is more flexible, but the latter was clearly (at least to me) good enough for OCaml.

Yeah, this is definitely something that I need to take a good, clear look at, which is not happening right now. I should go to bed as soon as I can.

Good night.

Coding 2023-09-20

2023-09-20T04:00:00-04:00

I've written a bunch of Impliciula code. It's not ready to show off yet, but it is making me question whether the whole "make code more like sentences than mathematical notation" concept is actually feasible. Some of these lines are over 100 characters, so one thing I need to salvage this is some idea of where line breaks are workable from a parsing perspective, and helpful from a punctuation perspective. I just tried something that looks helpful, but it's still all a little breathless.

Syntax aside, something occurred to me about semantics:

If I keep up the whole gradual typing idea, then code that isn't being accelerated should need to specify the resume behavior of effects at definition time. I think. Like, yes, when the code is being accelerated, it's helpful to guarantee that the fastest/smallest paths allowable get baked in, but in the dynamic regime, the handlers need to be tracked dynamically, so...

Maybe the way to do this is to just allow using _ when specifying what kind of resume behavior an effect has...

In any case, I guess I should focus on the effect syntax, because once I have a version of that that I'm comfortable with, I can put out some more concrete stuff here and elsewhere.

Anyway, I should stop looking at my screen.

Good night.

Weekly Roundup 2023-09-19

2023-09-19T04:00:00-04:00

Wednesday: I asked for some help with Impliciula. As of now, there have been... no takers.
Thursday: I thought about a new project that I could start.
Friday: I was tired.
Saturday: I had brain fog.
Sunday: One of my ideas for dealing with the brain fog didn't work out, so we kind of tweaked it, and we'll see at some point.
Monday: I don't want to go into details, but it sucked, and continues to suck.

Next week, I might have it in me to sketch out standard library modules for Impliciula. It'd be nice if someone expressed interest, but maybe I can just inflict them on a Discord server or some part of the fediverse.

Diary 2023-09-18

2023-09-18T04:00:00-04:00

Well, I got some bad news today. Something that I thought was bad, was actually worse. Oh well. Nothing to be done for now, so far as I know.

Aside from that, I'm trying to balance "needing to take things easy" with "wanting to get work done on something". I think I did all right today, but I'll do better if I just cut things short now.

Good night.

Diary 2023-09-17

2023-09-17T04:00:00-04:00

Well, the ways I tried to improve things today didn't really work out, but we've got something slightly different that we're going to try later. Just need to take things a day at a time and not try to rush to the end, I guess.

Relatedly, I did get some other stuff done today, although I would have liked to have done more. Oh well.

I should stop writing and try to wind down.

Good night.

Diary 2023-09-16

2023-09-16T04:00:00-04:00

Ugh...

I really want to work on stuff, but my mind feels... tangled. Like a tightly wound spring that's rusted together.

I'm trying to think of ways to get out of this state, and there are some things I can try, but the only sensible course of action to begin with is to cut this entry off here and get some sleep.

Good night.

Diary 2023-09-15

2023-09-15T04:00:00-04:00

Tired...

Just wanna sleep forever...

Not gonna get to...

Good night.

Diary 2023-09-14

2023-09-14T04:00:00-04:00

Hm. This is awkward. I was coming up with stuff to work on later, but I don't really have much to talk about from that yet...

I mean I guess I can drop a few things in here... I want to retool Missable Mysteries, in part because I'm no longer vibing with the name. Ideally, I want something that is a little more up-front about its currently hidden agenda, which is to armchair-diagnose a celebrity with dyscalculia.

But I had some thoughts about math stuff I could talk about without having a weird hidden agenda, and I'm currently trying to put together an outline for "alternative representations of numbers, especially ones that easily express things that we don't normally think of as numbers". Like "a notation that treats numbers as a special case of matrices" or "this is an extension to (a subset of) the rationals, which breaks ordering, but not the way that complex numbers do". One thing I need to work out with this idea is how much theory to frontload...

Anyway, I've got nothing else in me for today, so it's time to wrap up.

Good night.

Coding 2023-09-13

2023-09-13T04:00:00-04:00

I've decided that I don't see a way forward for Impliciula without someone to discuss it with, so I put out posts on Cohost and Mastodon asking for anyone who's interested in discussing a weird programming language idea. If that describes you, please take a look.

Anyway, I spaced out until it was way late, so I'm not going to post anything else.

Good night.

Weekly Roundup 2023-09-12

2023-09-12T04:00:00-04:00

Wednesday: I looked over some of the fiddly corners of OCaml's syntax.
Thursday: I looked at Koka, specifically the syntax for defining effects. I don't know what most of it does.
Friday: I thought a little about how to handle effects.
Saturday: I chilled out because I was feeling terrible.
Sunday: I had some ideas about how to divide up the kinds of effects.
Monday: I had some thoughts about what effect syntax would be compatible with how I'm planning to make the syntax of Impliciula.

Next week, I'm not sure what I'll be focusing on.

Coding 2023-09-11

2023-09-11T04:00:00-04:00

Quick thoughts about how to do effects in Impliciula:

The form of the effects has to be visible at the handler definition somehow.
I think this "should" be done with something akin to a match construction.
And a perform builtin to dispatch the effect.
This does prevent the development of the val sugar.
It would be nice to be able to construct handlers as values.
However this works, it goes in a block construct...

These handler ideas sound about right, and I suppose the perform idea is about right...

Potentially effect interleaving could be handled by some form of newtype wrapper? That's probably not scalable, but I'm not sure. I don't fully understand the approach that the paper I was reading was taking, so I don't think I can come up with something comparable yet. I'll have to keep on reading that paper, I guess.

Anyway, it is very late now.

Good night.

Coding 2023-09-10

2023-09-10T04:00:00-04:00

It's late, but I want to see what I can do quickly to lay the groundwork for working on the effect system later.

The minimal effect system in terms of syntax is to simply have one way to define operations.

This is problematic for two related reasons, both deriving from the fact that I want expressive power on the level of Koka for this: Effects that strong use a continuation that can be called arbitrarily many times, which means that the straightforward implementation would have to basically use "heap frames" instead of stack frames, which would both make performance pretty bad in general, and complicate reasoning about behavior.

(Another issue with having such "simple" syntax is that it doesn't allow for specifying that a particular operation should be handled by a handler other than the nearest one.)

One possible course of action that occurs to me is to try to express this stuff in terms of modular implicits somehow. I'm not totally sure if that makes sense, but let's see...

Doing stuff with "normal" modular implicits means that you're relying on matching up an implicitly available module to the input types, and potentially that module could define functions with effects. One question I need to consider is whether it works to vary the effects in different versions of the module... I think that kind of thing at least needs to be considered with modules in general, so, just as a module can define a type, a module can also define an effect.

Okay, trying to get past that diversion, what matters is that a function that has an effect, is relying on its caller to provide a handler; that handler can be given various abilities to resume the computation. My quick read on what are possible amounts of resumption to allow is that the cases that are meaningfully distinct are "never, once, more than once". These can come together as:

Resumption is never allowed.
Resumption is allowed up to once.
Resumption is allowed an arbitrary number of times.
Resumption is allowed exactly once.
Resumption is allowed once or more.

"It must be resumed more than once" doesn't sound sensible, so I'm going to put it aside for now.

One of the things I need to understand here is, what assumptions I need to relax when considering multiple resumption...

And I think I need to read over the Koka documentation again to understand what questions I should even be asking...

Anyway, that's all I have time for right now.

Good night.

Diary 2023-09-09

2023-09-09T04:00:00-04:00

So, I had (or have) a migraine, and I took measures to deal with it. They seem to have worked, but they do mean that I lack the focus to write in any detail. We'll see what I'm capable of tomorrow, but I can tell that "trying harder" isn't going to do anything in the next hour, so, like, don't push myself.

Good night.

Coding 2023-09-08

2023-09-08T04:00:00-04:00

I had a bit more of a look at the paper I mentioned last entry, and I've come to the conclusion that I'm probably not going to get anywhere unless I start putting together strawman syntax and semantics examples.

Anyway, I am sick or something so I'm going to go lie down byyyyyyye.

Good night.

Coding 2023-09-07

2023-09-07T04:00:00-04:00

I'm looking at the Koka documentation on effects, and here are the way things look offhand:

An effect can be named.
An effect can be linear.
An effect can be recursive.
An effect can have a "kind".
An effect can be "in" a type.
An effect can define any number of operations, including zero.
An operation can be pub.
An operation can be val, fun, or ctl

...

A handler can define a finally clause, even though finally is also a function??? Like, Impliciula would be able to get away with this, because one would be spelled finally and one would be spelled finally::, but Koka has more conventional syntax, so I'm confused.

Anyway, I'm trying to read about the implications of all of this, but it's late and I'm tired.

Good night.

Coding 2023-09-06

2023-09-06T04:00:00-04:00

I decided to switch back to Impliciula, and there's a chance that I'll switch back again soon because... one sec.

So, because I've got rough equivalents for a lot of OCaml syntax together, I went through the language extensions to see what I need to figure out how to include. There are a few definite yeses, a few things that sound important, and a few things that sound mainly like the horrifying stuff that people figured out how to do with the coding: directive in Python.

I don't understand all of the extensions, in part due to the intensely abbreviated presentation in that part of the manual, but I've got initial notes on all of the extensions, even if one extension just got notes of "Yo what the fuck".

Now, I need to put some time into reviewing Koka's syntax and shoring up my understanding.

Note to self, document an "explicit subset" of Impliciula's syntax when the time comes, and then gradually explain how to abbreviate that. That seems like it would hopefully be less mysterious than the presentation of the Koka book. Although, maybe the Koka book just feels like that to me when I'm trying to use it as "the wrong kind of documentation"? I'll think about all of that later, starting with "how should defining and handling effects look?" (One thing I'd like to keep in mind is whether it's possible to somehow describe "effects that only propagate outwards", if that makes any sense. Like, if I want to use a comparison function that logs to somewhere.)

Anyway, I'm not getting anything else done right now.

Good night.

Weekly Roundup 2023-09-05

2023-09-05T04:00:00-04:00

Wednesday: I switched gears to writing, and described some of the stuff I'm working through with Sundered Empire.
Thursday: I continued on with that.
Friday: I did a bit more writing, and pondered whether I'm going to end up cutting a bunch of the stuff that I was working on.
Saturday: I mentioned some of the character traits I hope to avoid, and pondered what's necessary to avoid overcorrection.
Sunday: Travel, but I published the chosts.
Monday: Tired.

Next week, I don't yet know what I'll feel like working on. We'll see.

Diary 2023-09-04

2023-09-04T04:00:00-04:00

I didn't end up doing anything worth writing about today, so I'm just going to have another short nothing entry.

Good night.

Diary 2023-09-03

2023-09-03T04:00:00-04:00

We traveled today, and I don't feel great. But I decided to shove all my Cohost drafts out the door, so here's my Cohost. Have a look, see what you think, comment on it if you have an account.

I don't think I have anything else to say.

Good night.

Diary 2023-09-02

2023-09-02T04:00:00-04:00

Getting to some good stuff in writing, but I need to think some about where to take it next. I've got some travel and stuff coming up in the next few days, so maybe I'll end up switching gears for a bit again soon.

One thing I'll need to ponder is a way to give the original main character a chance to shine again. The (not-quite-drafted) introductory sequence has them doing cool stuff, and then being brought low. I'm going to need to figure out how to showcase their skills, because they're mostly way outside of a context where their skills would reasonably apply.

Like, this is pulp-inspired, and I'm trying to avoid the "hypercompetent outsider" concept, but they should be good for something besides moral support. I'll have to think about this.

Good night.

Diary 2023-09-01

2023-09-01T04:00:00-04:00

I got a bit more writing done. Once I get the lines I want written, maybe I should skip ahead a bit, because it feels a little like I'm spinning my wheels here.

Like, I'm concerned with whether the new stuff I'm writing is fulfilling to read, whether it's adding anything beyond a recap of what the reader recently read. I may be accidentally recreating the conventions related to the serialized nature of the kind of science fiction I'm trying to emulate. And, if so, is that a problem? Maybe I should try to figure out a way to release this stuff serially? I dunno.

Anyway, I am tired and I don't want to prolong this post.

Good night.

Diary 2023-08-31

2023-08-31T04:00:00-04:00

I haven't heard anything about not making us go into the office for, frankly, no good reason, even as the trend lines bend back upwards. This sucks.

I tried to distract myself from that by doing some writing. I've gone a but further forward in the Sundered Empire stuff, at least chronologically, but I keep on setting the characters challenges and puzzles that then I have to figure out how they'd tackle them.

Like, okay, someone shows up from space, and, through heroic efforts at communication, conveys to you that they were sent to do something, and whether they succeed or fail, both outcomes are bad. What do you do with that information? How do you determine whether they're running some kind of con? Suppose it seems reasonable that the outcomes would be, in fact, bad, but it's not clear whether those outcomes will actually come about.

I suppose also I'm worried that all of this caution might read as strange to some of the audience, but I think it makes sense, and I don't want things to go super-easy for the main characters. I'm definitely not going to take it as far as "and now they have a knock-down brawl", not least because the one from space needs physical therapy to deal with being-in-space deficiency.

I guess I'll just have to sleep on it.

Good night.

Diary 2023-08-30

2023-08-30T04:00:00-04:00

Well, I'm still thinking about Impliciula, but I don't think I'm up for the amount of documentation I'd need to read to figure out what capabilities it needs to have, and how they should be expressed.

So, writing. I have no idea how well I did on the scene I'm drafting currently, but it's not something I have much experience with, so that's interesting to me. Two characters are trying to communicate, but they have one language in common, which neither of them are entirely fluent in. One has a broad vocabulary but mostly uses the language informally, resulting in "uneducated-sounding" grammar. The other can remember what words mean after hearing them used, but as far as speaking, can only remember entire sentences meant to be used as cocktail-party witticisms.

I'm trying to go for a contrast between an expert in various matters who ends up talking "like a caveman", and someone who can sound "high-bred" but has minimal relevant knowledge. I don't know how well I pulled it off, but whatever. The one thing I still want to do in this area is to have a technobabble lecture in caveman-speak.

But for now, I need to get to bed.

Good night.

Weekly Roundup 2023-08-29

2023-08-29T04:00:00-04:00

Wednesday: I did some work considering how to get syntax highlighting for Impliciula in Cohost.
Thursday: I did some work specifically on the lexer needed for syntax highlighting.
Friday: I improved the lexer.
Saturday: I filled in some more gaps in the lexer.
Sunday: I figured out the answer to a bunch of technical issues with Cohost.
Monday: I decided to take a break from this, and work on something else.

Next week, I'm going to somehow blog about the stuff I'm writing.

Diary 2023-08-28

2023-08-28T04:00:00-04:00

After all of the Cohost stuff I worked out, I still need to give those drafts some editing passes, but I don't wanna... So, for this next week, I'm going to try to focus on writing, to take my focus off the Cohost drafts.

I'm going to need to work on having stuff to say about this writing, or the next week's worth of entries are going to be really boring. Oh well.

I'll see what I can do right now, then get this published.

The setting for the story I'm working on is called the Sundered Empire. Centuries before the story starts, the lines of communication and commerce in an interstellar empire were severed by a series of rebellions, dividing the empire into many smaller states. In the years since, the different states developed their own identities; when the connections began to be re-established, the different states refused to re-integrate. At the time the story starts, fully reunifying the empire is a pipe dream, a topic of idle conversation among the upper class. However, one ambitious noble comes across hints of lost technology that could obliterate the borders between the states, and enlists the services of a legendary thief to recover it. The trail leads to a planet that has been left alone since the time of the Sundering, where the thief meets and eventually befriends an accomplished polymath who leads a small tribe living in the jungle. Stuff happens, and then they discover the lost technology and turning it on makes everything more complicated and precarious.

So, um, that's the first book, and, spoilers, maybe, I dunno. My hope is that I can get the readers through the whole "legendary thief on a strange planet" plot without ever overwhelming them with the "former empire" backstory.

We'll see how I do with expanding on that summary in a few days.

Good night.

Diary 2023-08-27

2023-08-27T04:00:00-04:00

I've gotten the prototype highlighter working well enough, for now, so I decided to try to get its output usable with Cohost. After smacking my head against it a bunch, I found the things that need to be done to use Pygments with Cohost:

Use the "Lightbulb" theme, or a similar dark theme.
However, replace the line number colors with whatever is the "default" color according to the scheme.
Replace all linebreaks in the HTML output with <br> tags.

The result looks a little weird to me, but it should serve.

Anyway, I got distracted and it's late, going to bed now byyyyyye

Good night.

Coding 2023-08-26

2023-08-26T04:00:00-04:00

Okay, I'm not in the mood to make all of the decisions that need to be made, but I started messing with the lexer prototype, and let's see what's missing:

Strings
Numbers
Arithmetic operations
semicolons
arrow
something about lists?

I got drafts for most of that, except for lists. Also I should have comments, which is going to end up being a whole other thing...

Nevertheless, I think I'm at the point where I can put together short snippets and have some highlighting. And, trying it out with my color scheme and... Wow. That's terrible. Like I should maybe re-evaluate the color scheme I'm using for this blog, because "half the keywords, and the function invocations and variable names all get identical highlighting" is a ridiculous scenario to end up in.

I'll try to get something that makes sense tomorrow...

...

I got sidetracked, and tried to make the theme from scratch, and I ended up with something nearly as bad as my custom theme, from modifying one of the bundled ones.

I'm going to try to ask for help. We'll see how that goes, but also I need to get ready for bed.

Good night.

Coding 2023-08-25

2023-08-25T04:00:00-04:00

Okay, I put together the chart, and iterated on it a bunch to write the necessary regexes for the prototype lexer. It's missing a lot of stuff I'm going to want down the line, but I'm trying to pick my battles. And the regexes are totally normal, nothing strange here.

You wrote a biblically accurate emoticon.

Come on, (?::(:{SEGMENT}:)*:?) isn't that bad...

That is 100% an angel named, like, Ixushavel, with the caveat that nobody who actually knows Hebrew was consulted on this point. And attempting to copy the "derivation" that "justifies" that name indicated that Hebrew almost certainly doesn't work like that. Or possibly that easily-available machine translation is being pushed past its limits.

Anyway, I should be good to test whether this actually works now. And once I've confirmed the basic cases, I can shore up some of the details I haven't covered yet, and then get back to the CSS Weeds.

For now, I need to get to bed.

Good night.

Coding 2023-08-24

2023-08-24T04:00:00-04:00

Okay. I want to write a lexer for Impliciula. For that, I need test cases. Let's start listing things. In each case, assume the whitespace is needed but I don't really care how it lexes.

list-of: 'a: function-inline-right, type-variable
list-of: int: function-inline-right, name
nil: name
head: head and-tail: tail: function-inline-right, name, function-inline-right, name
for-all-e-in: list e: ::is-even: function-inline-right, name, function-inline-right, function-prefix-left
there-exists-an-e-in: list such-that-e: ::is-even: function-inline-right, name, function-inline-right, function-prefix-left

Pause here to note that I will consider goofy unicode abbreviations for those, so they're more like ∀-e-in::e:: and ∃-e-in::st-e:: (which have to be function-prefix-left, function-prefix-left unless I want to devise Cursed Tokens or if I don't indicate the side on the prefix form, and let it contain multiple parts, in which case it's function-prefix, and that above is function-prefix-left instead of function-prefix)

The problematic cases come in with partial evaluation. Like, if I want to apply for-all-e-in::e:: to the predicate, can that be for-all-e-in:: e: ::is-even, or does it have to be for-all-e-in: : e: ::is-even? I really want to make the former work, but that would seem to mean that function-prefix-right, function-inline-right, function-prefix-left is a valid call. I can't "simply" lex the freestanding colons separately because otherwise I think the inline-right, prefix-left sequence is instead something really messed up. I think I'd like to allow the prefix version to contain double colons, because the alternative is to just split it into a bunch of prefix-rights. Prefix-left can only be a single segment, and prefix-right and prefix-right-left end in double colon and can maybe contain multiple segments.

I'm going to need to make some kind of Punnett square for the token types I'm trying to come up with here. Not a Punnett square. Um... I dunno.

I'm going to wrap up, and doodle that chart while this publishes.

Good night.

Diary 2023-08-23

2023-08-23T04:00:00-04:00

The Cohost writing is almost done. It just wants some syntax examples. Which means I need to write the lexer. Which means I need to figure out if my current ideas actually lex.

The issue I need to work out focuses on two special values I need to handle. The simple part is that I want _ to be handled specially, but the special handling doesn't need to be in the lexer, so whatever. The complicated part is...

So, I'm trying to base the function syntax off of Smalltalk, sort of. And one of the "great" ideas I came up with was to use : not just as a terminator (and sometimes starting character) for parts of a function name, but to represent an unevaluated function using :: instead, so : is used to indicate partial evaluation. (The starting character version is because I decided that I wanted to allow having an argument before the function name. Smalltalk doesn't have to specially represent this because—OH MY GOSH, LOOK OVER THERE)

So, the idea is that there will be functions like :is-even which can be delayed by writing as ::is-even, and part of my uncertainty about all of this is how to put everything into a nice canonical form.

Anyway, it's too late for me to think about all of this, so I'm going to wrap up.

Good night.

Weekly Roundup 2023-08-22

2023-08-22T04:00:00-04:00

Wednesday: I complained about my laptop being messed up for over an hour, then made plans for how to get feedback on NABTO, now called Impliciula.
Thursday: I started drafting entries elsewhere.
Friday: We traveled.
Saturday: I did the stuff that we traveled for.
Sunday: I did some work on the drafts, but started encountering issues that I don't fully understand yet.
Monday: More drafting, and we traveled back.

Next week, I'm going to try to finish up the drafts.

Diary 2023-08-21

2023-08-21T04:00:00-04:00

All right, I did some more Cohost writing, and we traveled back.

The Cohost writing feels weird. It's been so long since I properly drafted a blog post ahead of time.

I'm going to need to find editors for this to force me to confront issues like "I'm not actually explaining anything". Maybe I can just let the initial versions be kind of bad, and revise them after publishing if anyone pays attention?

Either way, for now I need to get to bed.

Good night.

Diary 2023-08-20

2023-08-20T04:00:00-04:00

I slept way in today. I was able to make some progress on the chost drafts. I think with a few more days worth of work, I'll be ready to post them. Although, maybe I'll want to have syntax examples, so that's going to push the estimates out a bit.

Mentioning that sent me out into the CSS Weeds. The short of it is, I'm not sure how to make Pygments play nice with Cohost's CSS restrictions.

I got as far as "remove usage of CSS variables", but at this point the style still has all kinds of Problems. Like, something somewhere in the box model for what I'm trying to do is Too Chunky, but I haven't had the wherewithal to methodically track down what's going on there.

I'll try to work out what's going on there later, but for now I should get to bed.

Good night.

Diary 2023-08-19

2023-08-19T04:00:00-04:00

The day has been accomplished. In the end, I wasn't up for doing my own thing much, but whatever. I wrote a few more stubs for Cohost; at this point, I'm going to try to get all of the stubs filled in over the next week or so.

I don't have it in me to do anything else today, though.

Good night.

Diary 2023-08-18

2023-08-18T04:00:00-04:00

Traveled, not up for much. I kind of anticipated this, so I'm just going to chill. We'll see how tomorrow works out for me, but I'm not really sure what to expect. I assume we should have the end of the day pretty free, but maybe that's totally wrongheaded.

I really really need to just listen to some music or something, so I'm going to wrap this up now.

Good night.

Diary 2023-08-17

2023-08-17T04:00:00-04:00

I futzed around some with post drafts on Cohost. I don't have anything to put up there yet, but with a few more days of tweaking (not necessarily these next few days), I should be at a point where I'm comfortable publishing all of my posts and angling for feedback.

I still need to decide what to do with the various bits of software that I'm going to want in order to support early discussion...

Good night.

Coding 2023-08-16

2023-08-16T04:00:00-04:00

Hhhhhhgghblhhhhhhhhhhhhh.

The laptop overheated or something earlier. Did Not Like That.

Anyway, I'm thinking about ramping up on Cohost with NABTO (which I'm probably going to end up calling "Impliciula"), and there are a few steps I feel like I need to take:

Extract the code highlighting stuff from the blog stylesheet
Put together a prototype Pygments lexer for what I have currently of NABTO/Impliciula's syntax.
Get this hosted somewhere. This isn't totally straightforward because I don't want to use Git, so I'm personally thinking either a Mercurial forge or the Pijul nest.

Flashbacks to the time you listed every SCM you could think of except Git.

Let's be fair, I also didn't list BitKeeper.

Anyway, I'm a little out of it, possibly because resurrecting my laptop is kind of stressful.

Isn't your working theory that you just need to wait for it to cool off, which admittedly seems to take a really long time?

Stop being right; it doesn't help.

Anyway, the idea is that I'd figure out how to incorporate that stuff into the workflow of something like prechoster, so I can drop code blocks into my chosts. Hypothetically, I could also drop the raw HTML into the ReST here, but, ehhh.

I lack the focus to get on this right now, but I guess I need a venv with Pygments installed, and then I can create a custom lexer, and I can invoke that... somehow. Looks like I want to base it off of python -m pygments -x -l your_lexer.py:SomeLexer <inputfile>. I bet I could write a script to handle all the dispatch stuff... Anyway, time to wind down.

Good night.

Weekly Roundup 2023-08-15

2023-08-15T04:00:00-04:00

Wednesday: I tried to figure out how I want the effect system in NABTO to work. I don't think I came to a solid conclusion.
Thursday: I fiddled some more with the syntax I want for NABTO.
Friday: I went into some proper detail about the weird syntax experiments I want to try out.
Saturday: I didn't have anything to post about.
Sunday: I went into more detail on the syntax ideas, and pushed them further into the language.
Monday: I figured out another puzzle I was having with the syntax.

Next week, I guess I'll keep on with this. I'm considering moving my posting habits around a bit to make it easier to get feedback on some things; I'll update the footer if I decide to go through with that.

Coding 2023-08-14

2023-08-14T04:00:00-04:00

I've punted on whether to have rec (or "recursive"), because I wanted to focus on how to keyword modules and modules types. I need some kind of keyword for various things to have them inline. That also includes functions, so maybe I can come up with some insight there?

Well, I got an idea for inline functions, but I don't feel any closer to having a handle on "struct" or "sig". It's like... these don't correspond to a basic grammatical form, more like a poetic or rhetorical structure?

Maybe the right way to look at them is as some kind of block structure, in which case the inline form would just be structure {...} or signature {...}. Which, given what I've gone through thinking about this, seems reasonable.

Well, time to get to bed, and maybe think about rec, I dunno.

Good night.

Coding 2023-08-13

2023-08-13T04:00:00-04:00

I'm messing a bit more with the Smalltalk-inspired stuff, and I don't think it's going to deliver on its goals of readability, unless I can come up with a better type definition syntax than let type list-of: 'a be nil or cons: 'a onto: list-of: 'a. Either I need to figure out a reasonable way to make that read more like prose, or I need to come up with a more modest goal. Or just scrap the whole thing.

To be honest, I do like the aspect of putting in the parameter's purpose next to its usage. So, maybe I want to focus my efforts on make the usage conform to some standard, and don't worry so much about the definition...

Oh man, I just tried annotating types on a function definition, and I think I actually really like what I came up with.

In that case, now I want to focus on match/switch/case statements. Right now, it's generically ML-ish, which doesn't particularly mesh with the pseudo-English going on around it. Let's look at the parts involved:

A value, which should be named like a noun.
A list of pairs.
The first part of each pair is a destructuring operation, which should be a type constructor, and I've just realized that cons: _ onto: _ doesn't look like a noun phrase. Heck.
Anyway, the second part can look like any part of speech I come up with, as long as it's consistent for a given statement.

Thinking about it that way implies that, instead of ->, I should be using the second keywords from the various forms of definition. That's no use without something for the beginning to coordinate with it, though.

Hm. If you squint at it hard enough, like maybe really really hard, a match construct kind of looks like an anonymous function that gets pipelined into.

...

Okay, I've got an idea. It's weird, but at this point, it's commit, or be committed. The new syntax I'm trying out is when _ is _ (be|do|then) _, and...

I like it.

I am going to have so much trouble convincing people this isn't an esolang.

Anyway, I'm going to wrap up for tonight, and ponder something else for a bit: should I have an equivalent to rec from OCaml, or just not, like in Koka?

Good night.

Diary 2023-08-12

2023-08-12T04:00:00-04:00

I seem to have been out of it all day, so I guess I don't have anything to write about. I've got some stuff set aside to look at later. But for now, I should properly wind down.

Good night.

Coding 2023-08-11

2023-08-11T04:00:00-04:00

Yesterday I mentioned having some weird ideas to try out for NABTO's syntax. I've been thinking about them some more, and I figured I could try describing them.

Here's the basic idea: the same underlying semantics can be represented with different syntaxes. See, for example, Hy, which allows writing Lisp code that ultimately runs on the Python VM. So, I can experiment with how different surface syntaxes work to express the concepts in NABTO without worrying that it will compromise the underlying semantics in some way. The basic rule I think I need to follow is that a language concept has to either map to an idea that I want to take from another language, or it needs to be possible to implement it as a local syntax transformation.

With all that said, researching for NABTO has exposed me to a bunch of different ways to express concepts, and, like, they're fine, but... So, what set this off is how unfamiliar the type construction syntax in OCaml is, but then I found myself considering the corresponding syntax in other languages, and the way that many widely-used languages represent function application... Maybe it's just because some of my coworkers don't seem to have a problem writing horrifyingly long parameter lists, but I think the usefulness of making functions in programming visually resemble functions in mathematics breaks down when simply enumerating the parameters (not even explaining what they're for) looks like it would take a full paragraph of prose.

It sounds like you're advocating for making certain types of code harder to write because of a personal preference. You're going to ruffle some feathers with that stance.

Well, I don't know how effectively I'm "advocating" for anything, but I'm not saying nobody should ever write a fifteen-parameter function. I just think that they should work to justify the contribution of each parameter to the cohesive concept that the function represents. f(x, y, z) is good enough as it goes, but I've found myself longing for something a little more verbose. More talk. Perhaps a... small bit more talk?

I don't see many people using Smalltalk, and I'm not sure if Objective-C got traction for any reason besides "On Mac, you kind of had to use it", so the idea I'm dancing around, of expressing a function as a phrase, with the parameters slotted into predetermined gaps in the function's name, may alienate some people. To them, I say, "Don't worry! NABTO isn't object-oriented in the Smalltalk sense, so this is probably going to also alienate the people who were nodding along before."

Let's get some examples. Suppose you have a list called lst. In Python, you'd get the length of it by calling len(lst), or maybe lst.__len__() for... some reason. In OCaml, length lst or lst |> length. In Koka, length(lst) or lst.length.

I am pondering something like length-of: lst. In this, a typical function would be named as a noun phrase with one or more gaps within it. I'm currently leaning towards omitting articles for the sake of brevity. If I put them in, they'd end up functioning as a means of distinguishing values (have articles) from types (don't have articles). But for the moment I'd rather see how well things go with a mild headlinese influence. ("For moment, rather see how things go with mild headlinese influence"?) Once I have the mental image of code mimicking prose more closely, I imagine stuff like replacing the = in let ... = with be, and changing out Koka's := operator for set ... to.

Not all functions "return nouns", though. A function returning () must be some kind of verb, and a predicate is very much like an adjective. Therefore, suppose that we allow to ... do as an alternative to let ... be, and allow a sort of "one-argument pipelining" with an is operator. So, like if 4 is even:: then ..., and even:: could be defined like when n is even:: then .... Or iff n is even:: then .... I'm not seeing good sugar for something like element: e is-in: l, so maybe that needs to settle for let element: e is-in: l be ..., unless it's permissible to use the non-is syntax in iff ... then as well.

One obstacle to this idea is that I'm not always sure what noun to use for various higher-order function concepts.

Also, how do you intend to handle pipelining in general, if your syntax is so inflexible? Or default arguments, for that matter?

For pipelining, I have an idea that amuses me: let somewhat-large be fibonacci-number: 1700; factorial-of: that. That is, using that in all code like _ in the Python REPL. And speaking of _, increment: number by: _, literally letting the function definition fill in the blank.

There is a lot of syntax to try and fit into this paradigm, but I want to see how far I can push this.

For now, I'm going to wind down.

Good night.

Coding 2023-08-10

2023-08-10T04:00:00-04:00

I'm still tweaking the code samples and pondering how I want effect handling to work. I ended up with this weird imbalance where most of the code looks like OCaml, but the effect handling looks kind of like Koka, and my initial goals for the language and my thoughts on the effect handling would be better served by having it the other way around.

Part of the problem is that I was basing things off of the Koka code samples, which have heavy doses of syntactic sugar. My intuition is that I'll make something much more coherent if I focus on the subset of Koka that does stuff like use brackets instead of indentation.

There's also the somewhat "fun" aspect that I'm trying out syntax that neither language uses, which ends up necessitating entirely new forms of syntactic sugar to handle capabilities from the source languages that don't translate cleanly.

(With the ideas I'm trying to pull in, I kind of imagine NABTO coming together into some kind of negaverse Raku.)

I'm not going to do any more tonight, but I think the right way forward is to chart out the relationships between the different syntactic elements of the languages, and figure out what purposes they're serving.

Good night.

Coding 2023-08-09

2023-08-09T04:00:00-04:00

I wrote some sample programs in a possible syntax for NABTO. There are various things that are likely to need to change:

I haven't considered how to integrate types into the syntax.
The effect handling syntax is mostly taken from Koka, and it's not gelling well with the OCaml influences.

I think what I need is something like... There are a bunch of type constructors that package record/variant/whatever types together with information related to resumption, and defining an effect consists of somehow associating a record type and a constructor together, and handling an effect consists of having a modified match statement that unpacks the resumption information. (Maybe there's some kind of automatic casting between different effect constructors?) Anyway, to perform an effect, the code needs to construct the data type associated with the effect, and then the runtime somehow determines which kind of effect wrapper to use... Although, that information would be ambiently available at runtime... Doing that would foreclose a number of desirable optimizations...

If I want to get the optimizations, then I need to be able to make a statement something like "from a type level, this type is guaranteed not to capture beyond this level of detail".

Wait... Maybe there could be an overloaded function, and then defining an effect can be done by setting up a modular implicit that associates the data type to the specific effect construction machinery. I'm not sure what the right way to approach this from an ergonomic perspective would be, but I think this can work. It conceals some weird footguns, but I think people would need to try to find them, so it's probably fine.

This has been a productive night, and I should wrap up.

Good night.

Weekly Roundup 2023-08-08

2023-08-08T04:00:00-04:00

Wednesday: Scheduling...
Thursday: I made some breakthroughs with NABTO.
Friday: I assume there was nothing in here because of the Scheduling again.
Saturday: The problem here wasn't the Scheduling, but just that I didn't manage my own time well.
Sunday: I got a bit more fleshed out about NABTO.
Monday: I went into more detail on NABTO, but didn't commit to much.

Next week, I'm going to try to get more interesting stuff done with drafting NABTO.

Coding 2023-08-07

2023-08-07T04:00:00-04:00

One thing that I'm not sure about when it comes to a standard library for NABTO is, what kinds of data types are useful to have? My intention is that NABTO should be usable for writing scripts to execute from the shell, simple low-traffic internet-connected operations like a small IRC bot, stuff like that. And I'd like it to scale up to bigger stuff, but I think it makes sense to nail down the behavior, and then investigate what it takes to scale it.

Anyway, there's algebraic stuff I definitely want, like booleans, optionals, lists, results, eithers (OCaml draws a distinction), maps, sets, arrays, tuples. And there definitely need to be facilities for manipulating bytes, and some kind of good interface to files full of bytes.

But I also want to have the ability to deal with text and numbers, and these abilities raise a number of questions that people feel very strongly about:

Should there be multiple numerical types, or just one? (When there's just one, it seems to me like it's usually a 64-bit floating point type.)
If there are distinct types for whole numbers and floats, should the whole numbers have a specified size, or be unbounded?
If there's a bounded type for whole numbers, should there be multiple sizes? Should there be signed and unsigned variants?
If there are different floating and whole number types, what should be required to convert between them? What about converting between different flavors within those categories?
I probably want to represent rational numbers somehow. What representation should I use, and should it be in the core library or not?
Assume for the moment that text is represented as utf-8 by default. (There are some libraries that I should look into, that I think would translate interestingly into a module system.) Should strings be indexed to retrieve bytes, code points, graphemes, or should they simply not be indexed at all? Keep in mind that none of these categories will always correspond to "the width of a grapheme in a fixed-width font" in a nice way.

My thoughts on this so far:

It would be good to have facilities for using NABTO as a calculator (floating point or rational (and algebraic, if I can understand the work that's been done on algebraic representations)), and for reasoning about discrete chunks of memory and data (integers, possibly both signed and unsigned)
I actually do have some opinions about integer operations: I think I like how Python handles integer division; that is, always rounding down, sign of non-zero remainder matches the sign of the divisor. That is, the semantics. I'm not overly attached to the // syntax.
It would be good to have some way to get text into or out of a file, but that's not always what's going to be happening, and different types will probably be involved.
Maybe I want to try to replicate Python 3's thing where instead of comparing some instances directly, it takes a "key" function to convert to a type that is simpler to compare. (At least sometimes. Can't do that all the time without ending up in an infinite regress.)
Oh geez I forgot that the behavior of changing the case of text is locale-sensitive.
I don't know if it would help with that, but I guess I should look into how Swift does things.

Anyway, it's late and I want to wrap up.

Good night.

Coding 2023-08-06

2023-08-06T04:00:00-04:00

Here's what I've decided makes sense as a way to develop NABTO:

Figure out what the standard library functions should look like. I'm currently thinking somewhere between OCaml and Koka, but focusing on OCaml, making functions uncurried and row-polymorphic in effects, and furthermore using multimethods where sensible.
Figure out what I want to do for stuff like arithmetic and format strings.

That's all I've got for now. It's best if I get to bed now.

Good night.

Coding 2023-08-05

2023-08-05T04:00:00-04:00

Well, I'm still not in great shape to write anything, but I really want to figure out how to pitch the concepts behind NABTO.

Unfortunately, I got distracted doing research, and maybe the answer is "I need to do a bunch more research before I can talk about this stuff persuasively."

Anyway, it's late and I'm tired.

Good night.

Coding 2023-08-04

2023-08-04T04:00:00-04:00

Tired and distracted. My plan is to finish this post quickly, and take down physical notes on NABTO. But mainly, I need to get some rest.

Good night.

Coding 2023-08-03

2023-08-03T04:00:00-04:00

Another rough day, but I stumbled across what was a missing piece of the puzzle in NABTO's "design". Here are the three things I want to try to build it up around:

An effect system that works with the type system (like in Koka)
Modular implicits (like in this paper)
Gradual typing with some kind of sensible runtime semantics for the dynamic type

That last goal very obviously complicates the design and implementation of the other two. My feeling on this is that a language with explicit effects and modular implicits in an ML-style module and type system would be extremely interesting, but I want to see if it's possible to have gradual typing too.

(One other thing I've been rolling around in my head is the concept of marking an effect as "must be performed".)

There are... a lot of things that would need to be investigated and reconciled to see if all of this even makes sense, but now I feel like I have a specific goal to work towards. Stuff of note:

Koka vars, unlike OCaml refs, interact with the effect system.
I want effects, though not necessarily all effects, to be arbitrarily resumable, which means that the runtime needs to be written really carefully, especially in conjunction with gradual typing.
Modular implicits offer an alternative way to lay out some fundamental standard library modules, so that needs to be looked into.
OCaml also has a bunch of language extensions that seem useful for NABTO, but I don't fully grasp the tradeoffs and limitations involved.
I don't know what I want function calls to look like.
I want to see how stuff like Trio's design would work with Koka-style effects.
And generally the kinds of utility libraries that languages have.
Also I need to know more about Perceus.

Thinking about all of this, it kind of seems like my best bet for actually accomplishing anything here is to figure out how to articulate the vision I'm going for in a way that would convince people who, um, actually understand this stuff to put work in.

I'm going to get ready for bed now. Until I get some inspiration for how to write about this stuff persuasively, I can mess with parsing, or maybe some cursed Lua I put together recently.

Good night.

Diary 2023-08-02

2023-08-02T04:00:00-04:00

I am too tired and out-of-it to have anything to write for today. I'm sort of thinking of taking a break from the OCaml stuff, because there is, in fact, more schedule fuckery.

I'm messing around with other things right now, and there's nothing to write about those currently, so I'm cutting this off here.

Good night.

Weekly Roundup 2023-08-01

2023-08-01T04:00:00-04:00

Wednesday: I put together some ideas for internal data structures for an Earley parser.
Thursday: I decided to take a break from Earley parsers to mess around with Math Stuff that seems like it "should" work better in OCaml than in Haskell or Rust.
Friday: I redid the Math Stuff a little, and confirmed a few of OCaml's capabilities.
Saturday: I set out a bunch of work for myself on the Math Stuff.
Sunday: I decided to take a break from the Math Stuff to mess around with the Earley parser code.
Monday: I tried to refresh my memory about which types need to exist for the Earley parser internals.

Next week, I'm going to either power through these weird internals, or get distracted with something else.

Coding 2023-07-31

2023-07-31T04:00:00-04:00

I did other stuff today, but I did just now jump back into the Crafting Interpreters stuff. I'd mentioned that I wanted to implement the "dot" from Earley parsers as a zipper list. (Regular reminder that I'm messing with Earley parsers because I'm specifically avoiding what Crafting Interpreters actually says to do.)

Remembering what comes next:

I need to put together the composite type for representing an item using this stuff, including writing a comparison function.
I need to put together the composite types for representing the work queue items, and write comparison functions for them.
I need to sketch out what translations of previous iterations of the scan/parse/complete code will look like.

For now, I need to wind down and get to bed.

Good night.

Coding 2023-07-30

2023-07-30T04:00:00-04:00

Okay, I'm at the point where I need to take a break from this whole weird side project, and also to read some actual literature on constructivism. For now, the stuff I'm trying to put together is sprawling into something that doesn't make sense. Maybe by reading up on this stuff, I can come up with some helpful primitives.

This post, surprisingly, already took too long to write, so I'm going to just call it.

Good night.

Coding 2023-07-29

2023-07-29T04:00:00-04:00

I'm tired, but here's what I've worked out from reading Wikipedia and messing around:

There are five-ish basic structures I want to mess with to start with.
Each of these structures needs a few of its own versions of things:
- A "witness" module type that constructs a contradiction-checking function from components of the structure.
- A "witness of" functor that combines an instance of the structure type and the witness type.
- Functors that construct structure instances out of other structure instances; these functors should themselves provide functors that map from attestations of the components to new attestations for the composite.

The attestation concept has a lot going on, and I'm trying to figure out whether it needs specialized components as well, or if it can be general like some of the support code. An attestation, in my informal thoughts on this, has:

A structure (module, necessary for typing)
A property (module, necessary for typing)
A witness function (probably in a module; derived from structure and property, but I think not in a generalizable fashion)
The witness module provides an unreduced specimen type; there must also be a partially reduced specimen type
Fully reduced specimen type
Mapping from unreduced specimen type to list of partially reduced specimen type
Function to test reduced specimen types for a contradiction
Mapping from partially reduced specimen type to fully reduced specimen types

Probably more, there's so much in there... And some stuff that maybe can be moved out of the module interface. I think a bunch of this needs to be structure-specific, so I'll have to figure out how much of it can be factored out.

Okay, that's... a lot. I'm going to wind down and try to take it easy for a while.

Good night.

Coding 2023-07-28

2023-07-28T04:00:00-04:00

Okay, I let things go late tonight, so I'm going to try to just jot down some thoughts.

After the entry last night, I redid things a bit. Now, there's a single module per property (ish), and a given property and (currently) magma can be combined in a functor to produce a module that determines whether a collection of values in the magma represent a contradiction of the property. There are at least two directions that I need (want?) to take this.

What does it look like when I make the structure more complicated? The obvious place to start here is to single out a value of the magma's type, and to devise a property check that the designated value is an identity element.
Constructing these predicates is not the end goal. My gut tells me I want the ability to take the attestations for the property of some modules, combine those modules into a compound module, and also combine those attestations into a compound attestation.

I've been trying to mentally line up the different parts that could be involved there. Here's one idea that just popped into my head, and I'll lay it out, pop into the playground, and say whether it seems reasonable:

Assume that, when there's a compound or derived property at play, the user knows about it, so it's fine to do some one-off stuff.
A compound module functor knows how the component modules get put together, so it also knows how to put together corresponding attestations. (It also has the knowledge to do more elaborate stuff, like combining structures that carry out different operations on the same underlying set.)
So, if I'm using modules to represent both algebraic structures and assertions about them, does that mean that my "combine structures" functor can define a "combine assertions" functor?

...

Oh, yes.

:) (glitchy)

Ha ha ha... Yes

*Ahem*

I'm still getting a handle on what OCaml will and won't let me do. When I was first messing around with this space, I was trying to accomplish things in a way that very much did not fit with what OCaml lets you do, and it kind of seems like there isn't a way to make it do that, because the syntax that expressed my ideas already meant something different. So it's really gratifying to see a way forward to accomplishing this stuff.

I can try to do this stuff over the next few days, and see what falls out. For now, I'm going to wrap up.

Good night.

Coding 2023-07-27

2023-07-27T04:00:00-04:00

Okay. So. I could keep on with the Crafting Interpreters stuff I was sketching out yesterday. I could do a lot of things. Like take another tack for encoding Math Stuff.

So I was reading an article on constructive mathematics, and one point it made was that the kinds of limitations that constructivists place on mathematical reasoning can be related to the kinds of limitations that software has to be written under in order to... do anything. One example I'll develop is, it's not obvious to me how you'd establish that a particular property holds for every member of a set. So let's flip it around. Let's reason about what we can do with a contradiction of a property.

Well, if the set is, handwavily speaking, "simple", then, not too much, really. But if the set is built up from smaller sets in some manner, then it should be possible to reduce the contradiction into one that applies to the smaller set. If not, then there's something wrong with the reduction or the composition, and we can report that still.

Let's see if I can put this together in a way that makes sense to anyone, and hopefully to me. Let's have an informal class of types called elements. These elements are used to witness contradictions to a property. Something like:

type 'a total = Total of 'a * 'a
type 'a commutative = Commutative of 'a * 'a

(* I tried defining a "binary operation" sig, but I just confused myself. *)
module type Magma = sig
  type t

  val op: t -> t -> t
end

module type Witness = sig
  type t

  val is_contradiction: t -> bool
end

module TotalWitness(M : Magma) = struct
  type t = M.t total

  let is_contradiction (Total (a, b)) =
    try
      M.op a b |> ignore;
      false
    with _ -> true
end

module CommutativeWitness(M : Magma) = struct
  type t = M.t commutative

  let is_contradiction (Commutative (a, b)) = M.op a b <> M.op b a
end

I doubt that I got all of this exactly right. Like, the specific operation should probably be encoded in the witness type or module... Let's see if defining composition enlightens things any.

module MagmaProduct(M1 : Magma) (M2 : Magma) : Magma = struct
  type t = M1.t * M2.t

  let op (a1, a2) (b1, b2) = (M1.op a1 b1, M2.op a2 b2)
end

All right. There's a slight problem there with the module type specification turning t abstract, but, eh. I'm getting tired, but I'm going to try to articulate the bigger problems here. The key thing missing is the code to isolate and express the errors. That needs to build on the code above, or something like it.

At this point, I have some ideas for totally reworking this, but it's too late right now for it to be a good idea to try to write them up.

...

I tried working on it outside of this entry, and just confused myself. Time to get ready for bed.

Good night.

Coding 2023-07-26

2023-07-26T04:00:00-04:00

I do have some other ideas that I'm turning over in my head to see how I could express them in OCaml, but the current result there is "seemingly pretty poorly". Like a bunch of really messed-up functors...

So, let's get back to Lox. By which I mean, the stuff that Crafting Interpreters specifically recommends against doing, in the effort to implement Lox.

I'm trying to figure out the right structure for the state sets and the "work queue". Because I want to augment the state sets with additional information, let's temporarily call them something else, like "cells". When we're working with cell #n, then we need access to possibly every cell from 0 to n inclusive, as well as "the current work queue" and "the next work queue".

There are several operations that call each other recursively.

Adding an item to a work queue consumes the old version of the queue, and returns a new one. Additionally, it updates the cell corresponding to the work queue...
Predicting an item consumes from a work queue (consumes and returns a new version), and updates the corresponding cell
Scanning an item consumes from a work queue and the current token and updates the corresponding cell
Completing consumes from a work queue and potentially any cell, and updates the corresponding cell.

Here's what my gut says to do to implement all of this:

Have a pair of cell and work queue.
Have a pair of those.
Do not add the "current" cell to the array until it is done being processed; special-case getting it in the completion code.
Have each function operate on the pair of data structures, taking it in and returning it.
Maybe define some kind of fixpoint helper to replace all of the loops.

All of this gives me a reasonable target to shoot for later, so I'm feeling good about it now. I'm going to wrap up for now and mess with the other ideas on paper.

Good night.

Weekly Roundup 2023-07-25

2023-07-25T04:00:00-04:00

Wednesday: I got some code written for nullability detection in the parser.
Thursday: My laptop wouldn't turn on for like half an hour, and has since refused to elaborate. I did not like this.
Friday: I got some more code for the nullability checks written. It didn't look great.
Saturday: I tracked down some bugs in the nullability checks, through code inspection.
Sunday: I tried to clean up the code. I do not know if I succeeded.
Monday: I started planning out a format for storing information about the parsing process.

Next week, I'm going to try to flesh out and implement those plans, and maybe find something else to mess around with in OCaml.

Coding 2023-07-24

2023-07-24T04:00:00-04:00

I was doing stuff besides working on coding today, so I'm going to just drop a few notes on implementation stuff:

The logic for constructing a parse forest requires the index of the start.
The straightforward way to handle this is with an array, but I'm skeptical of the performance characteristics.
So I'm still messing around with whether it's possible to implement something decent using list. Like some kind of system of adding up indices so it's actually pointing back instead of counting from the beginning. There are two immediately obvious drawbacks to this approach: it sounds tricky, which isn't great when I'm trying to figure out the actual main implementation at the same time; and I don't actually know if any of it would help.
For parse forest construction, the useful indices would be the left-hand-side and the start index.
For actually constructing stuff, I think the relevant thing is completion, because the work structure can have any required format. To carry out completion, we need to look for a non-terminal after the dot, which is a completely different arrangement of data.
Maybe instead of "just" a mapping from left-hand-side to start index to sets of zippered right-hand-sides, I should have "that, plus a map from non-terminal-after-dot to 'whatever the format is for inserting into the work queue'"

I think that's enough to work with later. For now, I'm going to properly wrap up.

Good night.

Coding 2023-07-23

2023-07-23T04:00:00-04:00

Hm. I spent a few hours moving code around trying to make things "more readable", and I think I made some improvements to start with, but now it's just kind of, like, sloshing around?

Earlier I was looking for some kind of set of guidelines for when to use stuff like |> and @@, and I didn't have any luck, but now I feel like I really need some guidelines like that, because it's all just getting kind of weird now.

Anyway, it's way late, so I'm getting to bed finally.

Good night.

Coding 2023-07-22

2023-07-22T04:00:00-04:00

All right, it's late because I watched a movie that was strange, but probably should have been stranger somehow?

Anyway, let's take a look at the nullability code. Here is the premise: some nonterminal symbols are known to be nullable, and the rest are not known to be nullable. I have a set representing the symbols known to be nullable, and for the symbols not known to be nullable: a map of maps of sets of sets. It maps from a symbol that is on the right-hand-side of rules, to the symbol that is on the left-hand-side of such rules, and the set that it maps to is of sets of right-hand-side symbols not known to be nullable.

Working from the outside of the update_nullable code path...

It starts by converting the list of the right-hand-side to a NSet.t option containing all of the nonterminal symbols if there were no terminal symbols. If this is None, then there's no work to do and it passes back the set and map unchanged. Otherwise, it takes the set and passes it to the next function in. I think I just noticed a bug. I'm taking a set difference in the wrong place. Just move it sooner, and... Okay, that should work.

Okay, so it filters out all elements of the right-hand-side set that are already known to be nullable. If there's still something in the set, then that needs to be added to the map: Every element in the set needs to be added or updated in the map, by adding or updating the left-hand-side key, to make sure that it points to a set containing the new set. And if there's nothing in the right-hand-side set, then the course of action is to mark the left-hand-side as nullable.

Marking a symbol is nullable is accomplished with a recursive function that pulls and accumulates work from and into a set. If the set is empty, it has no more work to do and passes back the other argument unchanged. Otherwise, it pulls one element from the set, and, hm. It's getting late and I need to be better rested to grasp this code. I think I'm missing a call to NMap.remove, and I think I just put it in the right place. But the quick and basic idea is to remove the newly nullable symbol from the sets in the sets in the maps in the map, as well as from the maps in the map. At that point, if any sets of sets contain the empty set, then the corresponding left-hand-side needs to be added to the work set.

I mean, this all seems to make sense, but I'm really tired, so I don't know that I'm trustworthy about this. I should get to bed.

Good night.

Coding 2023-07-21

2023-07-21T04:00:00-04:00

Okay, I've potentially got the nullability detection code working properly. There are a lot of moving parts, and I suspect a lot of the code I threw together isn't idiomatic. Like, I managed to get one statement indented nearly halfway across the line. And some function applications that I think are in "the wrong order" from an allocation perspective. All sorts of code that can be cleaned up, if it's doing the right thing in the first place.

So, I can forge ahead with trying to handle parsing now that I at least have the representation of nullability in a usable state, but also I want to try to validate that my various ideas for this code are correct, and that the code as written implements them, and then look into simplifying things.

At the moment, I'll do neither, because I need to get to bed.

Good night.

Diary 2023-07-20

2023-07-20T04:00:00-04:00

The whole schedule change thing continues to be pointless. I think I might have some ammunition to get out of it, but we'll see.

Right now, though, I'm a little rattled because my laptop randomly shut down earlier today, and I'm just like "not again".

I've got some ideas for stuff to do trying to work on code written in OCaml, but like I said, rattled.

Hopefully my laptop is better behaved tomorrow. I did try to reduce some of the load on it.

Anyway, I don't have the focus to stay up any longer.

Good night.

Coding 2023-07-19

2023-07-19T04:00:00-04:00

I started specifying and implementing what's required to incrementally detect nullable rules. One hurdle is that I'm informally writing this out in a mostly imperative style, but I'm implementing it all functionally, and sometimes the gaps between the two approaches get... wide.

Complaining about this was enough of an impetus to try and get the implementation right before I could manage to explain what the issues were, so this is probably close to done, but there are still various bits of bookkeeping to handle.

I can't handle them right now, because it's late. Let's see what I'm up for tomorrow...

Good night.

Weekly Roundup 2023-07-18

2023-07-18T04:00:00-04:00

Wednesday: Not a great day.
Thursday: I tried to figure out one of my previous attempts to implement Earley parsing.
Friday: I looked at the module types I'd written, and tried to figure out how to improve them from a usability standpoint.
Saturday: I made some progress on simplifying the grammar modules.
Sunday: I moved on to considering the internal types relevant to the parsing operation.
Monday: I realized that I need to handle nullable symbols, and investigated a possible way to incrementally detect nullable symbols.

Next week, I'm going to keep on with that schedule stuff that's messed me up, and hopefully argue against it. Aside from that, I'm going to try to get the nullability stuff in a sensible state.

Coding 2023-07-17

2023-07-17T04:00:00-04:00

Now that I have some ideas for the types involved in the actual parsing part of the Earley parser, I've got a better handle on visualizing its operation, and I've concluded that I do, in fact, need nullability checking. Which is a shame. Loup Vaillant has an explanation of what that's supposed to solve, but I'm going to try phrasing it in terms of my "trying to teach this stuff to myself" code structure.

Basically... Start with the three operations: scanning, prediction, and completion. Now, ignore scanning, because it inherently operates on two distinct state sets, so it's unproblematic in this context. Going over prediction, it does operate on just one state set, but once a prediction has been done, there is guaranteed to be no more work. Now, completion is a little more subtle, because it's unproblematic if the item being completed was started in an earlier state set, which is guaranteed to be the case... unless the symbol is nullable.

There are a few ways to deal with this:

Retry completions until nothing changes, making sure to bring in predictions as those are added.
Determine the set of nullable symbols immediately before parsing.
Determine the set of nullable symbols incrementally, as rules are added.

I kind of don't want to do things the first way, because I suspect that that would end in some weird redundant loops.

So, I need to either have nullability calculations right before parsing, or incrementally. I want doing it incrementally to be a slam dunk, but I suspect that it's not. Basically, if you add a single rule, the immediate question is whether it renders the left-hand-side symbol nullable, but if it does, then every non-nullable symbol has to be checked for whether making that first symbol nullable makes that other symbol nullable. Jeffrey Kegler's note on the nullable stuff presents a linear-time algorithm for detecting nullables. I'm going to try to walk through what doing it incrementally would look like.

So, there are going to be several auxiliary maps. These maps only need to concern themselves with rules that have no terminal symbols. One map from LHS to rules. One map from RHS symbols to sets of rules (including LHS I think). A set of nonterminals with empty rules. A mapping from every non-terminal in the LHSs to whether that is a nullable symbol, false by default.

At this point, things diverge sharply and I kind of have to wing it. I believe I'll have to re-derive the time bounds of what I'm about to propose, if I want to know what they are.

When adding a rule:

If there are terminal symbols, ignore it.
If the rule is empty, add it to the set of empty rules. Actually, this step and data might not be needed.
If the LHS is already marked nullable, ignore it.
If the RHS is made up only of nullable symbols, mark the LHS as nullable, and then, iterate over every rule with the LHS in the RHS, to see whether any of those rules now have only nullable symbols in the RHS. If so, mark the associated LHS as nullable and recurse.

I'm honestly missing what anything besides the RHS map and the nullability map is accomplishing in this, so I'm going to see how far I can get with just those.

One thing I note is that, once a symbol has been marked nullable, there's no reason to check any rules with that LHS, so it could be efficient to remove such rules from the RHS sets. Potentially there could be a different way to store the RHS data other than literally, like as sets of symbols not known to be nullable. Then, when a set gets to zero, mark the LHS as nullable and cut down on stuff. You know, stuff.

I think this is enough to get started with tweaking the representation. Looking over it, I'm highly skeptical that what I have in mind is worst-case linear, but it shouldn't lead to any huge degradations in grammars that don't "need" this stuff.

One more pass at trying to make sense of this:

To mark a rule nullable, check the RHS table. It should be safe for the RHS table to contain tables from the LHS to the sets of symbols not known to be nullable. When a symbol is marked nullable, then that means that it needs to be removed from every other entry. For each other RHS symbol, put together a set of LHS symbols to check. If any LHS symbol is associated with the set of just the nullable symbol, then that symbol has to be marked nullable on the next pass. For LHS symbols with other sets, those sets represent RHS symbols that need to be updated; doing so is guaranteed not to produce more nullable symbols.

I'm still not sure about the time bounds, but I think I can get this to make sense, so I'm going to take down some notes on how I need to change up stuff, and then get to bed.

Good night.

Coding 2023-07-16

2023-07-16T04:00:00-04:00

I went over the parser code for Lox a bunch, and I think the interfaces make sense now. At this point, the functor requires two simple modules, instead of three with a complicated relationship between them, so at minimum, the code is simpler now just because I'm not trying to use features that I think I have some serious gaps in my understanding of.

At this point, the only code missing for the basic parser case is... the code that actually handles parsing. The hurdle to clear there is that I need to figure out the types required to handle the "items". It's like I need to consider every phase of the parsing process simultaneously to figure out what data is actually needed. But, like, there must be some way to break this down...

The state sets are stored in an array. The sets themselves should be NMaps containing sets of partially completed rules, and start indices. I'm not sure whether some of these will need to carry information about semantic actions, or if I can safely just leverage the grammar to retrieve them. Let's assume the latter.

It may be the case that I want items derived from a successful scan to carry the matching terminal symbol. That could also be in a tuple with the map, or simply not stored within the array. Because of those latter two options, let's assume that it's not stored inside the items. My idea for the structure of the items is to have two lists of symbols like a zipper list, and for the queued form, to take the Cartesian product with the relevant symbol type. This means that each queue has a different type, which will match up with the action associated with that queue.

Now that I understand the queue types, I should hopefully be able to write them out properly and get the flow between them correct. Unfortunately, to actually implement this, I'm going to need to either get really clever with pure functions, or bite the bullet and write a bunch of imperative stuff.

I'll think about that over the next few days or whatever.

Good night.

Coding 2023-07-15

2023-07-15T04:00:00-04:00

All right, I've given things some thought, and here is what I've concluded: If you're making modules and types for grammars the way I intend to (which is clearly not the only way), then it's going to need a bunch of gnarly compound types, but crucially, they're entirely built up from just two basic types that the user has to pass in. So, if we give those types as arguments in OrderedType modules, then that relieves the user of constructing a bunch of modules to pass in.

The next thing I need to think about is how to handle the whole error-wrapping idea. Is it reasonable to say "If you want an error-correcting grammar, then use the special error-correcting grammar constructor, which will give you back a module for a non-error-correcting grammar, and then you can pass that into a function on the higher-level module"?

Maybe... maybe the error-correcting version should have a different relationship between its input types and the compound types in the grammar, compared to the basic version.

Like, what's the right choice between:

A module that requires another similar module
A module that provides another similar module
A module that secretly uses a similar module in its implementation?

And now I'm realizing that a lot of these types should be abstract, which will hopefully make the code shorter overall. Like, the compound types that the code needs to know about:

The grammar itself. This should be a map from non-terminals from an implementation perspective, but all the consumer needs is an empty grammar function, and a function to add rules.
Rules within the grammar. The user needs a way to construct these, which requires the ability to create instances of the non-terminal and the matcher types.
Semantic actions. The user needs a way to construct these, which requires the ability to destructure the token type.

So, ultimately, this requires three public types, and one abstract type. Two of the public types need a comparison function to implement the various compound types.

I think this is enough to fix up the interfaces around the grammar, but right now I need to get to bed.

Good night.

Coding 2023-07-14

2023-07-14T04:00:00-04:00

I stared at my old code some more, and I'm starting to get a handle on what exactly I was doing. It looks like the difference between using a set and a dict was just a matter of some cases "wanting" to cache the result of a computation.

I'm thinking about how best to handle the idea of yet another type to put into a Set or a Map. I must be doing something wrong, because all of this boilerplate is really unpleasant.

I wonder if I can get away with turning all of this "map" stuff into functions from keys to values, because that would cut out a lot of this nonsense. Or maybe I should be trying to define just a few Maps in the interface, and constructing the implementation out of OrderedTypes?

Today was another exhausting day, so I'm going to wrap up for now, and try to figure out what's a sensible way to do this stuff tomorrow or over the weekend.

Good night.

Coding 2023-07-13

2023-07-13T04:00:00-04:00

I still haven't got a handle on the new schedule stuff (honestly, I'd like to figure out some way to just be exempted from it, because I don't see what the point is). Anyway, I'm going to try to take some quick notes on Earley parser stuff and have that be my post.

So. My attempt at a Python implementation has a few moving parts. One of these is called add_item. It supposes that if an item is already in an Earley set, then the next action for it must be queued up, and it accomplishes this by first checking, then queueing, then adding. The queues are represented by a set[Item] (for completion) and two dict[Item, Symbol]. An Item is a combination of a Rule and an integer index, which could be obnoxious if I translate it directly, because I think that means a bunch more modules. The different queues are pulled from using different functions that don't return a value, they just mutate the queues. Now, a close port of the Python logic would actually not be The Worst, because the queues don't leak anywhere, but I would like to see if it's possible to avoid using mutation.

I think the missing piece in understanding how to update this is just to take some notes, because reading the ipynb file in vim is... probably not the best way to get a handle on this. Anyway, I will regret it so much if I don't get ready for tomorrow, so I've got to be done now.

Good night.

Diary 2023-07-12

2023-07-12T04:00:00-04:00

Schedule change results so far: terrible; awful. Tomorrow, I'm going to try changing the schedule in a different way, which should hopefully leaving feeling a little less bad.

To try to offset the bad vibes I was dealing with, I picked back up some writing I'd been meaning to get back to. It's pretty good stuff, and so far it's still doing what it's supposed to, so I think that helped a little.

Anyway, part of that schedule change is going to involve getting up earlier, so I should go.

Good night.

Weekly Roundup 2023-07-11

2023-07-11T04:00:00-04:00

Wednesday: I got scanning for Lox working in OCaml.
Thursday: I tried to figure out what I'd want for implementing parsing for Lox.
Friday: I speculated a bit about what types might make sense for parsing. (Some of the types, did not, in fact, make sense.)
Saturday: Trying to actually use those types didn't work out or at least was difficult, but I forget the details.
Sunday: I tried to work through how to write the relevant type signatures.
Monday: I didn't go through the details, but I did a bunch of cleanup work that obsoleted the speculation I did the day before.

Next week, I've got some schedule changes to deal with, so I'm not totally sure what I'll be up for.

Coding 2023-07-10

2023-07-10T04:00:00-04:00

All right. At this point, the types are... less wrong. I'm highly skeptical that I've gotten this stuff to a completely idiomatic state. I'm now getting back up to speed on the right way to actually carry out the parsing.

Looking over my previous disorganized notes, I think my plan is to try to do things kind of naively, which means putting together the equivalent of a do-whole loop, but also needing to figure out how to translate the non-local references in the Python version...

Although, if I end up feeling up to writing Python for hobby stuff again, the insights I had around representing this stuff in OCaml could be helpful as far as writing type annotations that make sense.

Anyway, it's late and I'm tired, so I'm cutting this off now.

Good night.

Coding 2023-07-09

2023-07-09T04:00:00-04:00

All right, let's see if I can put something together quickly.

...

And I tried actually using the stuff I wrote before, and I've concluded two main things.

One is that I'm pushing past my comfort zone a bit in terms of the OCaml features that I'm trying to use. More importantly, the code as I've written it is just kind of a confusing mess.

I just now tried to tidy it up a little, and... doubt. I'm going to try to push forward and see if there's any kind of, like, obstruction.

So, the current thing I'm trying to put together is a parse function. It starts with a grammar and (currently) a sequence of nonterminals. The first thing it needs to do is create a ('t, 'o) item NMap.t array. That array should start off containing an empty map. It's only empty because I'm trying to break this up into steps.

Aw crud. I got a few steps in before realizing that one of the types is wrong. I need a container to go in the middle of one of these horrifying types, and the code is still so bad that I think doing this technically qualifies as an improvement. So, here's the puzzle... I need to take the terminal matchers and make them suitable to use as (part of) keys to a Map. To make that work, the functional values have to be removed. I think this can be accomplished by creating a module to hold the function definitions, and make the actual matchers into normal variant types.

...

Okay, I've started doing that, and now part of the code is a lot cleaner, but I'm pretty sure the rest of the file is just covered in red lines because I only started this. I'm going to leave it there for tonight.

Good night.

Coding 2023-07-08

2023-07-08T04:00:00-04:00

I tried to put together a signature for the grammar stuff for Earley parsing. I've tweaked it a few times, and I'm almost certainly going to tweak and change it a lot more as I figure out what's actually reasonable and what's needed to actually handle the task of parsing.

I think I have what I need to test it out by actually trying to write the functions, but I'm not up for that right now. We'll see about tomorrow.

Good night.

Coding 2023-07-07

2023-07-07T04:00:00-04:00

All right, let's see what it takes to handle Earley parsing, in terms of types.

type 't terminal_matcher = { match_ : 't -> bool ; produce : () -> 't }

Might change the input to that second function. And I think it should not be required that the produce function only produces values that satisfy the predicate.

Anyway.

type n; module NMap = Map.Make(struct type t = n; let compare = compare end); type ('t, 'o) grammar = n * (((n, 't terminal_matcher) Either.t list * (('o, 't) Either.t list -> 'o)) NMap.t)

Without the language tools, I don't know which parentheses are needed, or even if I got that all right. So, let's see if I can explain all of that.

With the terminal_matcher, I'm maybe kind of bending the rules for "what is a grammar", because the usual description is in terms of terminal symbols, which are more-or-less associated with a particular field on the token type in this context. So, the "sensible" thing in my mind is to have a type that relates to the token type, and write helpers that express "match this terminal symbol" in terms of what structure the token has.

Now, to handle the structure of the grammar, I need to have the nonterminal symbol type as a module-level type instead of a variable, so I can use it to instantiate a Map. This map is used to hold the majority of the information in the grammar, with the remainder being the initial symbol next to it. Within the grammar, each value is a pair of a list of nonterminals and terminal matchers, along with the semantic actions, which are functions that combine lists of their output type and terminal symbols.

The expected definition of the action functions is a little dynamic, because I couldn't see a way to encode all of the type information in an existential type so that it doesn't interfere with the ability to put all of these different signatures into one type of slot.

Oh yeah, I defined a record type to hold two functions for the matcher, instead of just aliasing to one of the types, because I'm trying to plan ahead to having a functor that replaces n with n option and provides the ability to convert from the wrapped module type to the wrapped type for this module, but with extra output stuff.

A missing piece of the puzzle is that the grammar modules need to define a parse function that takes a grammar and a list (or something) of tokens, and the error-correcting version almost certainly needs a custom parse function to try to avoid all of the pathological parse behavior that I'm pretty sure adding the error correcting would introduce, at least the way I'm thinking of doing it. (Other people may be built different.)

I think I should be done for now, and think about this more over the weekend.

Good night.

Coding 2023-07-06

2023-07-06T04:00:00-04:00

Okay, I'm trying to move on to parsing now, and I need some time to figure out the right way to handle it.

I'm still on the idea I had previously of "let's learn something besides recursive descent", in part because I think trying that in OCaml would lead to an utterly monstrous let rec that would only get bigger as I go.

So, I'm back looking at Earley parsers, and I think I've got an idea of how to put the types together. However, I want to consider error correction possibilities. From doing some research, I think I want to do something like this:

First, have a grammar definition, including semantic actions.

Then, the following:

A transformation of the semantic actions to a form that includes an error score.
New rules for matching "junk" where each token matched increases the error score.
Terminal matchers need the ability to create "synthetic" tokens in the absence of proper input, which signal somehow that they're not present in the source.
Create rules in which the terminal matchers are replaced with matching zero or more junk, and their argument to the semantic action creates the synthetic token.
The junk rules need to create something that the modified matches know to discard.
Some combinations of this stuff.
Probably the additional data should include error details so those can get reported.
Bias advancing in the parse without errors (maybe using effects to create continuations or something?) so that working code doesn't end up generating a massive error forest just in case.

Like prioritize by number of errors, and then by index.

Okay, this sounds workable, but I'm not sure. I'm going to have to sleep on it.

Good night.

Coding 2023-07-05

2023-07-05T04:00:00-04:00

Traveled back, and messed around with algebra stuff in OCaml. But let's ignore that for now, and see about getting scanning written for Lox. What do I have so far...

I have a function that attempts to produce a token given some source string and a control data structure. It returns an updated control value, and a token option. The equivalent code in Lua relied on a helper function that acts similarly to filter_map id, so, um, let's assume I'm using that, and I need to define a function that produces a token option Seq.t or something.

Given all of that writeup, I've managed to write the desired iterator, wire it in, handle the effects, and...

Something, somewhere in it is broken. The problem currently seems localized to the number function. I'll see if I can fix that up quickly.

...

Got it. I forgot to copy over everything, and I ended up with an off-by-one error. Now it all seems to work perfectly.

I'm going to wrap things up for tonight, but now feels like a good time to sum up how I feel about OCaml so far:

The normal programming with looking at types and matching everything up is great, basic meta-programming with the module system is fine, trying to push the limits of "obvious" when it comes to meta-programming just ends in confusion so far.

I'd like to figure out if there's some way to avoid that confusion, but I'm going to keep assuming for now that it's a result of a deficit in my understanding. Anyway, it's late now.

Good night.

Weekly Roundup 2023-07-04

2023-07-04T04:00:00-04:00

Wednesday: I kept on struggling with algebraic structure stuff.
Thursday: Ditto.
Friday: I decided to step away from that whole thing, and revisit Crafting Interpreters.
Saturday: I decided to use a new, shiny feature for Crafting Interpreters.
Sunday: I decided to directly adapt some of my old Lua code into OCaml, and felt like it ended up weirdly similar.
Monday: I had good luck continuing on with that stuff.

Next week, I'm going to see how much further I can go. I made more progress today that I haven't written up, so I anticipate I'll get a clean build soon; maybe then I'll discover some weird bugs as a result of the conversion.

Coding 2023-07-03

2023-07-03T04:00:00-04:00

Travel.

But also, a bunch more work with OCaml. I'm writing helper functions and finding places that can probably be cleaned up later in the code that I'm writing. The code feels pretty good to write. We'll see later how it feels to read.

I put writing this entry off way too long, and I'm not going to draw it out.

Good night.

Coding 2023-07-02

2023-07-02T04:00:00-04:00

I ended up deciding to try to patch together the scanning code from one of my Lua versions. While I can express a bunch of the basic primitives fine, as I get to somewhat higher level stuff, I'm hitting imperative constructs, and, like, while I can use those in OCaml, I want to try to avoid them.

Like, let's see. I've defined a function called str. Its OCaml version will take a source string and a control record. The first major thing it does is enter a while loop that advances the state of the control record in various fashions.

By laying that out, I seem to have gotten a little further in implementing this stuff, and fixed a few issues along the way. I hadn't quite grasped that OCaml's effect system allows you to pass effects around, and invoke them separately from constructing them. This makes sense in retrospect, but it's not something I currently need, so I had to change some code.

Anyway, big thanks to ocamlformat for taking my stream-of-consciousness one-line monstrosities and making it so that at least I can read them.

Finally, I should be about done for tonight. I've got a rough template for how to port the rest of the code, and then I can try to give it a nice interface.

Good night.

Coding 2023-07-01

2023-07-01T04:00:00-04:00

The more I thought about how I wanted to handle errors from inside the Lox interpreter, the more I was inclined to try using the experimental effect system in OCaml. It's like, yes, I'm pretty sure OCaml absolutely would allow me to make a module level ref and twiddle it back and forth, but I'd rather scope the ref to the run function and make the caller handle the output. So, I've written the helper functions that produce the effects, but I need to do a bit more work to be able to get the effect handler into the run function.

Hopefully, if I skip ahead and around a bit, I can put together the proper code soon. Ideally, without going to the next chapter before I get all of this handled.

I can work on that later. For now (or, half an hour ago...) I should get ready for bed.

Good night.

Coding 2023-06-30

2023-06-30T04:00:00-04:00

I took another look at the code that I was planning to... "take inspiration from", and either one specific part of it is subtly non-idiomatic, or I don't understand idiomatic OCaml. I'm going to assume the latter. In any case, I'm now looking for some other place to start messing around in.

After I pondered it a little, something occurred to me that made a lot of sense: Lox from Crafting Interpreters. I've gotten relatively far with it before, mainly hindered by the fact that I wanted language features that are in OCaml, but not so much in Lua. So, if I'm focusing on the early parts of Crafting Interpreters, I can focus the effort of learning on "Okay, I know how to do this thing in a bunch of languages. What's the OCaml version?"

...

All right, I've gotten a few things done, but it's about to get weird. I'm going to need to get error handling right, and that sounds like a thing to work on later.

Good night.

Coding 2023-06-29

2023-06-29T04:00:00-04:00

Okay. I had a look at how the different structures with some group properties relate, and I don't think I can "just ignore" stuff like associativity, because of how it seems to interact with divisibility.

Trying to state what I actually want to accomplish here:

Any given structure should be able to specify "laws" that its operations satisfy, both as a run-time specification of a test, and as a compile-time specification of "capabilities that the module provides". Because this doesn't necessarily relate to a single function or value, I want to somehow tag this information to the module itself. Which means having something else in the struct to hold that static and run-time information. I can't see any way through this that doesn't involve somehow constructing an intersection of types, be they module types, object types, or something else.

One thing that might help constrain what I'm looking for is to consider what the tests look like. If I'm requiring that every property can be "pulled off" of the module where it definitely applies, then it needs to carry a reference to that module. For now, let's assume that I probably shouldn't do that.

I'm going to try to revisit one of my original ideas... Hm. Wouldn't work, but now I'm curious about module constraints and extensions...

Ugh. Nothing is coming together with this, so I'm looking at how other people have done stuff in this area. I think I want to just swipe how other people have done this, and try to extend that with tests of the laws.

Maybe I can try that out later. For now, I need to wind down.

Good night.

Coding 2023-06-28

2023-06-28T04:00:00-04:00

Well, I messed around in Peglin, but I'd rather work with OCaml stuff for this post. Let's see about some of the stuff I was thinking about last time.

I decided to start off by trying to make some of the currently not-too-important code I wrote "actually work", and I ran into trouble. Here's the deal. Any algebraic structure that includes a magma can potentially have that magma be commutative. This isn't too important to the main... does anyone call it the Group Theory Cube, or something like that? But it's definitely something that comes up. For example, the trivial group and the cyclic groups are commutative. So, if I'm all like, "I want to mark the loops coming out of this functor as commutative", or whatever, then I need to be able to construct a signature for "a loop, but commutative".

I'm not sure if the way I'm currently trying to express this even makes sense in OCaml. Like, the stuff I'm reading kind of makes it look like "stick stuff onto a random signature" isn't an intended thing, in part because the syntax I'm trying to use is very similar to the syntax for a completely different concept. So, maybe I need some other way to mark "properties that only become apparent by calling the magma operation multiple times". The first thing that I thought of is "something with phantom types", but that doesn't seem to have solved what I think is the actual hard part, of "some magmas can be associative, some can be associative, some can be both, some can be neither". And worse, "what if somebody else thinks of a property to check?"

Maybe if I can figure out how to express "this set contains such and such member" as a type-level assertion, I can turn this idea into "there's a set of properties".

... Or maybe I can swing something with object types. Like, structural typing gets me "I can sort of have type lattice stuff", but I think it's going to discard that information pretty aggressively. I guess the big thing I need to think about is "how do I think this information is even going to get consumed?" Like, I can figure out how to write tests for these properties, but I'm not sure how to express "these properties are needed for this context".

I'll have to think about it, and then commit to something for now.

Good night.

Weekly Roundup 2023-06-27

2023-06-27T04:00:00-04:00

Wednesday: I laid out some ideas for stuff to do in OCaml.
Thursday: I tried to explain what I'm trying to accomplish in OCaml.
Friday: I thought some about how to write the tests for the OCaml stuff.
Saturday: Peglin...
Sunday: I had even more thoughts about how to write the property-based tests in OCaml.
Monday: Didn't do much I guess.

Next week, I'm hopefully going to get more done with OCaml or writing or something.

Diary 2023-06-26

2023-06-26T04:00:00-04:00

I was kind of out of it today, and we were doing stuff that I don't have anything to write about, so, eh.

It sucks that I don't have more to write for now, but I don't.

Good night.

Coding 2023-06-25

2023-06-25T04:00:00-04:00

I mostly took things easy today. I did just now mess around with writing OCaml modules representing some simple groups. From doing this, I concluded that I need to tweak things some more.

Here's the deal. I wrote a functor that creates a cyclic group given a module containing an int. However. If I want to generate tests for such a module, I need to let the functor that generates the tests know what the order of the group is.

I think there are two ways to accomplish this:

Create a new signature specifically for cyclic groups that exposes the order.
Create a functor that generates the relevant cyclic group, next to the module containing the tests.

I think I'm going to need to reach out to someone who knows what they're doing, but my gut reaction is that it's probably more helpful in general to expose the order as part of the interface of the module. My reasoning here is, suppose we have hypothetical other metaprogramming based around these kinds of functors. I think it's a similar amount of effort to implement either option, but it seems more helpful for multiple forms of metaprogramming to all act on the same module, rather than generating their own copies. (My recollection is that each invocation of a functor is a distinct module, even if the input modules are identical.) And this would theoretically allow people to supply their own implementations of, for example, cyclic groups. Not obviously helpful, but I don't see what preventing it would accomplish.

Anyway, it's late, and I should wrap things up.

Good night.

Diary 2023-06-24

2023-06-24T04:00:00-04:00

Okay. I poked a bit at the OCaml stuff, but didn't get too much done. Some writing. Some Peglin.

*ahem*

Kind of a lot of Peglin. If I get a few more good runs, I should get another orb unlocked. It's not a good orb, but it's another orb.

I'm not thinking of anything else to write, so, oh well, calling it here.

Good night.

Coding 2023-06-23

2023-06-23T04:00:00-04:00

I just sketched out the eight basic signatures relevant to group theory. The next thing to look into is the twelve functors that are apparently necessary to represent "forgetting parts of the structure". Most of these should be really easy. (I guess there can be a few more, but, eh.)

After all of that, I can start thinking about functors along the lines of "here is a generator function associated with this structure" and "here are the tests for checking specific properties", and then "here are the tests for checking combinations of properties".

Once I prove that stuff out with simple things like permutation groups, symmetry groups, and modular arithmetic, I can try to build out more elaborate structures, at which point I should have everything I need to work on stuff like geometric algebra.

Just pondering this a little, but I guess the test functors would need something like another set of signatures that just refer to the particular functions in use, and they can use that to be like "whatever module you pass in needs to express these capabilities".

I'm not sure, though. Maybe it's sufficient to say "we compare the behavior to this simpler structure that does test for the property". But where does that leave stuff like commutativity?

I'm going to see if these questions are any less vexing tomorrow.

Good night.

Coding 2023-06-22

2023-06-22T04:00:00-04:00

Okay, yesterday in the summary, I promised to motivate my efforts to represent algebraic structures in OCaml. The motivation is a little abstract, but I'm going to try anyway.

One common aspect of mathematics is to build complex concepts on top of simpler ones. In some cases, it's essentially possible to swap out the specific "simpler concept" for a different one with similar properties. To get even more hand-wavy, there is a type of algebraic structure called a "field", which is basically "something that you can do high-school algebra to", and "being able to do high-school algebra to something" is a prerequisite for being able to do college-level algebra to it.

In other words, if you have a bunch of fields, and a bunch of things that do something to fields, you can snap them together arbitrarily like legos with no physical form. Basically, I want to really grasp stuff like geometric algebra, and, um, things that Norman Wildberger thinks would be easier to teach than current curricula, and also have the tools to pose questions like "what's the weirdest thing I can come up with that's still technically a 'triangle'?"

I'll see when I'm up for actually working on this; I was kind of tired and out of it today, so maybe tomorrow, maybe later. For now...

Good night.

Coding 2023-06-21

2023-06-21T04:00:00-04:00

Okay, I'm kind of out of it right now, but I'd like to work through my ideas for other stuff to try with OCaml. One concept that ML-style modules seem well-suited to representing is that of an algebraic structure, which is something that Missable Mysteries would have/will end up covering.

Kind of want to change the name...

I've tied myself in knots trying to fit an algebraic structure (admittedly a somewhat complicated one) into typed Python. I think doing it in Rust would require a bunch of newtypes in the general case, because Rust traits are all about "the implementation" of a concept, and some of the really interesting stuff happens when you're explicitly manipulating multiple groups derived from the same set.

Whereas in OCaml, a specific algebraic structure would be a module, a kind of algebraic structure would be a module signature, and a mapping between algebraic structures, such as "forgetting" parts of a structure, would be a functor.

This is slightly short of what I want, because I'm not sure if there's a good statically-analyzable way to handle the axioms of an algebraic structure. So, what I want to see if I can do is to express axioms in a form that a property-based testing library can handle. I searched around a bit, and found qcheck.

I think there are two basic concepts that need to be defined together to write the kind of tests I have in mind:

An axiom corresponds to a test function.
An algebraic structure will have some associated types, which need to have some kind of generator to produce values.

I guess all of this is going to take a bunch more modules and functors. Like, given an algebraic structure with some signature, that needs to combine with a generator, which... stuff happens, and eventually a module pops out with a list of tests.

Properties to test include:

totality
commutativity
associativity
divisibility
invertibility
identity
distributivity
forgetting properties in different orders results in structures with the same behavior

I'm pretty tired, so I'm going to wrap this up now, and then sketch out exactly how I think this stuff should fit together.

Good night.

Weekly Roundup 2023-06-20

2023-06-20T04:00:00-04:00

Wednesday: Peglin and OCaml.
Thursday: Peglin and OCaml.
Friday: I worked a little on OCaml stuff, and included a digression that inspired me to somewhat improve keyboard-related stuff. That's not quite all the way done, but whatever.
Saturday: Some more progress with OCaml.
Sunday: Travel and Peglin.
Monday: Travel and sort of OCaml.

Next week, I'm going to try and make more progress with OCaml, and maybe branch out into some stuff that I can talk about without worrying about "spoilers".

Diary 2023-06-19

2023-06-19T04:00:00-04:00

And, we traveled back, and I'm still a little out of it.

I've been messing with some ideas for the OCaml stuff, and I'll try to execute on them later, but right now it's late and I need to get to bed.

Good night.

Diary 2023-06-18

2023-06-18T04:00:00-04:00

We traveled today, so I'm kind of out of it.

Nevertheless, I now have almost all of the unlocks in Peglin that seem... reasonable? I'm missing a seemingly very rare event combination, several Cruciball levels, and a relic that can only be unlocked in custom games.

The sensible thing to do is to try for the rare events while pushing Cruciball levels, since the custom unlock conditions look... silly. Like, that there is a meme run.

Or maybe I'll step away from all that for now, and do, like, an All Skull Orbs custom run or something.

Anyway, it's late and I can't think of anything else to say.

Good night.

Coding 2023-06-17

2023-06-17T04:00:00-04:00

Okay, I'm still not feeling great, so I'm going to make this quick.

I poked at my OCaml Cryptopals code today, and I'm getting it into a position that I think makes more sense, but it's going to be even better once I put together some functors. This is perhaps "ludicrous overkill", but I really really don't want to write two copies of this helper code.

(By the way I played a little Peglin and I can't tell if it was trying to unlock orbs or playing on the highest unlocked Cruciball, but Cruciball 7 is a pain.)

Anyway, I've got some more helper code to write. It's really pleasant putting some of this stuff together.

For now, I really need to get to bed, though.

Good night.

Coding 2023-06-16

2023-06-16T04:00:00-04:00

I wasn't up for much today, because I had a terrible headache for a lot of it.

I was able to start on some coding stuff though. I've made several attempts over the years to do Cryptopals stuff in Python, and I've solved the early sets in, as I recall, a few different ways. (I... don't want to actually look at the old code, because some of it is so cursed that I can't currently type like that, though that's admittedly more to do with not bothering to set up the proper keyboard-related software.)

I'm messing around with the code for the very first challenge. Basically, putting a lot of effort into being a very specific kind of lazy. Specifically, I'm not checking whether there are already libraries for what the challenge is about. And I'm trying to avoid boilerplate. This is going to make a bunch of the code really "clever", but that's kind of the point, so.

In any case, I still haven't had the time and focus to get this to a point where it all even compiles, so, you know, slow going with a new language. I'll just have to keep at it.

Good night.

Diary 2023-06-15

2023-06-15T04:00:00-04:00

Hoo boy, finally got the Balladin. Now I just need to figure out what random stuff I still need to unlock. I think there are a few things that I assume I must have taken that I actually haven't. The other side of it, of course, is events in the Castle that just. don't. spawn.

Anyway, I'm done with that for tonight. After I post this, I'm going to mess around with OCaml stuff...

Good night.

Diary 2023-06-14

2023-06-14T04:00:00-04:00

Still trying to get that really nice Ballwark build in Peglin, but at least I've grabbed a few new orbs.

Aside from that, I'm still trying to learn OCaml, and trying to work on Fair Fights. I made a separate blog for Fair Fights stuff, and I'll link it once there's anything there.

As far as OCaml goes, I'm most interested to learn the equivalent, where applicable, of common Python idioms, because if I try to just translate stuff over in a literal way, that... is unlikely to work.

Anyway, it's late and I should get to bed.

Good night.

Weekly Roundup 2023-06-13

2023-06-13T04:00:00-04:00

Wednesday: I played Peglin and did a little conlanging.
Thursday: Just all Peglin.
Friday: More Peglin, and messing with learning programming languages.
Saturday: More Peglin, and I wondered whether there's an "intended" time to get into custom starts.
Sunday: We went out and did a bunch of stuff, and I somehow had the energy for Peglin afterwards.
Monday: Yet more Peglin.

Next week, I'm not going to lie, there's probably going to be more Peglin. I've got a few other things I might have something to say about, but we'll see.

Diary 2023-06-12

2023-06-12T04:00:00-04:00

I was pretty out of it today. I unlocked stuff in Peglin, but I forget exactly what. A few relics that I was like "Now I can feel okay about never taking this again", and some like "Oh, I probably had a build in the past that would have benefited from this, but I forgot about the relevant stuff for the synergies."

Aside from that, I just mostly took things easy.

Messed around with my appearance a little. It's a work in progress, and I don't want to go into any more detail right now.

I don't remember if there was anything else worth talking about, so I'm just going to cut this now.

Good night.

Diary 2023-06-11

2023-06-11T04:00:00-04:00

Ooouuuugh...

I feel like a ping-pong ball that was in a dryer. That didn't stop me from unlocking a Cruciball level, but I don't really feel like doing anything else now.

Okay, time to try to relax.

Good night.

Diary 2023-06-10

2023-06-10T04:00:00-04:00

Whew, making more progress in Peglin. I ended up with a bunch of conflicting synergies, that basically compensated for the fact that they worked against each other by just always swinging high. Just like, "this is a Muscircle build!" *crits constantly*

One thing I find myself wondering is, is there an "intended" time to mess around with custom starts? Like, if I notice some weird interaction, should I just go for it and see what happens, even though I'm not yet halfway through the current crop of Cruciballs?

I guess I'll ask around somewhere. Right now, I need to wrap stuff up.

Good night.

Diary 2023-06-09

2023-06-09T04:00:00-04:00

It's possible that "unlock new orbs and relics", "unlock more Cruciball levels" and "unlock new characters" are goals that I shouldn't be trying to pursue all simultaneously. Oh well.

Anyway, I ended up deciding to focus on learning OCaml for coding, at least for now. My rationale is that the OCaml/Koka side of things is part of what I want to draw from for NABTO, with Python on the other side, and I want to have a better handle on OCaml, especially the module system, before I commit to any design decisions. Like, I think there are a bunch of issues I have with Python that would be addressed by having ML-style modules instead of class-based inheritance, but I'm encountering some trade-offs when I imagine going all-in.

I don't want to speculate too much, but it looks like the next thing I'd want to look into, once I have a handle on OCaml (and I'm just now starting out, currently), is Julia. But, like, one thing at a time.

*cough* GDScript *cough*

Two-ish things at a time.

Anyway, it's getting late and I want to wrap things up for tonight.

Good night.

Diary 2023-06-08

2023-06-08T04:00:00-04:00

Peglinnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnn...

Use your words.

That was a really spicy build I just pulled off. Sapper's Sack and a bunch of Ballwark orbs, as well as a bunch of other bomb relics. Basically, I was constantly building bunkers around myself to compensate for the fact that I kept blowing up literally everything on screen. I had no reliable in-battle healing. I would have lost at the end if I hadn't kept an egg.

:) (glitchy)

This is super fun and not at all terrifying. You all should try it.

In any case, I unlocked some things, but not the stuff I was really hoping for. And now, I'm going to make an effort to not stay on my laptop for another hour, so I'm ending this here.

Good night.

Diary 2023-06-07

2023-06-07T04:00:00-04:00

Peglinnnnnnnnnn...

I unlocked the Roundrel. Like, nearly six times over. The final boss got down to a quarter health and then instantly died to the ridiculous amount of spinfection I stacked on it.

Writing stuff for the blog made me remember about Fair Fights, so I messed around with some word order stuff. And, while writing this post, I got far enough for now, that I'm going to try to take what I've got so far, and put it together into names, see what I'm missing.

Once I've named, like, two monsters, I'll try to get back to game design and learning the tools.

Right now, though, I'm going to wind down.

Good night.

Weekly Roundup 2023-06-06

2023-06-06T04:00:00-04:00

Wednesday: I messed around with stuff for the Fair Fights conlang.
Thursday: I was kind of out of it. I contemplated speedruns that may or may not be interesting.
Friday: I tried to do a bit more with the Fair Fights conlang, but I was really suffering through the heat.
Saturday: I convinced building management to send someone to look at the AC. Building management did not convince them to show up that day.
Sunday: The HVAC issues got explained and worked around, just in time for the heat to break. Oh well. I messed around some more with the Fair Fights conlang.
Monday: I mostly played a bunch of Peglin.

Next week, I'm going to try to make some progress on the conlang, then put it on hold to learn GDScript. I'm thinking I'll also mess around with OCaml and BQN. That... probably won't be confusing.

Diary 2023-06-05

2023-06-05T04:00:00-04:00

So many things seem to be happening right now, and I haven't put in any real effort with that last Can of Wormholes level. I did spend a bunch of today being... not super great at Peglin yet.

I then ended up spending a bunch of time, instead of writing this post, trying and failing to figure out a simple way to convert a GitHub Pages site that uses a bunch of non-ASCII characters into a format that I can get on my e-reader. I may have something, but I'll have to look into it tomorrow. Because it's tomorrow now...

Good night.

Fair Fights 2023-06-04

2023-06-04T04:00:00-04:00

A big thanks to the HVAC technician who showed up (a day late, but whatever...) and both noticed what was wrong with the system, and explained it. This reflects really poorly on literally every other professional who has looked at the system over the past two years, but, again, whatever.

Anyway, let's ponder how to make the Fair Fights Conlang sound the way I want it to.

I've elected to give its syllables a modest collection of codas, but more extensive a set than what the cipher produces naturally. This is helpful, because I want to give the nouns some inflections based on gender and case. That's reasonable for a naming language, right? Because you have to know how you're using the nouns that you're naming stuff with.

They can't respond before you send the post.

Everyone's in agreement, then. Good.

Anyway, I want two or three genders here, and as far as cases go, the genitive for sure, dative is probably okay, and to round things out...

I am going to see how far I can get with ergative and absolutive. Make absolutive the "dictionary" case, and express the others via suffixes of some kind.

I seem to have run out of inspiration for the night, so I'm going to just cut this off and wrap up for now.

Good night.

Diary 2023-06-03

2023-06-03T04:00:00-04:00

I've almost 100%ed Can of Wormholes. Some of these puzzles are a bit much. Like, not figuring out what shape I'm supposed to make until I paste a screenshot into an image editor and mess with it. Just a bit more of this, and I should be done. I don't quite understand one of the achievements, but whatever.

Oh, what's the status of the air conditioning? WELL

Hold please.

DON'T KNOW WHY THEY WOULDN'T BELIEVE ME IN THE FIRST PLACE

Oh boy, there's some kind of glitching going on.

SAME ABILITY TO MAKE THE STATED TIME AS TO

We might be at this for a while.

COULD HAVE AVOIDED ALL OF THIS IF THEY'D DONE IT RIGHT TWO YEARS AGO

Oof. Bring it in, would you.

HUFF

So, that's been my day.

Oh yeah, last night, I came up with some phonotactics for the Fair Fights conlang, so I'll try applying those tomorrow to see if they help make things more palatable. Maybe I'll try to get started on that after I publish this, maybe not. Either way, I'm going to call this entry here.

Good night.

Fair Fights 2023-06-02

2023-06-02T04:00:00-04:00

I wasn't up for much today, and I won't be up for much tomorrow, because we're hitting high highs, and the HVAC in the apartment is [WORKING PERFECTLY FINE – source: people who aren't in here all day].

But I want to make some kind of progress on the Fair Fights conlang. I'm going to look for patterns in the stuff I've labeled as good so far.

Use of /u/
Detectable patterns in related words
Possibly, use of /ja/

Unfortunately, it's pretty late, so I'm going to try to cut this short. Looking at what I have, I think I need to put together proper phonology and phonotactics, because I skipped doing that due to the cipher. That might give me a vaguely more principled way to reject some of the stuff I didn't like.

Anyway, time to flop over in bed and stew in my sweat.

Good night.

Diary 2023-06-01

2023-06-01T04:00:00-04:00

The weather today wiped me out a little. I did some writing, but not for Fair Fights. Played more Can of Wormholes, which I've been playing a bunch; I'm currently at eighty-something percent, though all of the meaningful shortfall is in the final world.

... I wonder what the low% runs for Can of Wormholes look like. Like, you can skip some puzzles I think, although every puzzle that you use for wires you have to solve anyway, and it doesn't seem necessary to get every rocket to at least get started on the final world. Although, it's possible I'm overthinking this; I don't remember if I know the conditions for getting the credits...

Hm. I'm watching an any% run now, and I'm clearly going to have to look up what actually gates the ending. Looking at all of the stuff the runner isn't doing, I'm now not sure how interesting the low% routing can be.

Oh well. I'll see if I'm up for more stuff tomorrow.

Good night.

Fair Fights 2023-05-31

2023-05-31T04:00:00-04:00

I'm still categorizing roots, but at this point, I should be good to rate the many roots, and look for common patterns in the ones I like. Once I get some solid patterns in the parts of speech I'm considering, I'll start entering stuff into a database to hopefully be able to handle stuff like sound changes so I can get the conlang that I actually want.

For right now, I'm going to lie down and mess with the number system. It's getting late and I want to stop looking at my screen.

Good night.

Weekly Roundup 2023-05-30

2023-05-30T04:00:00-04:00

Wednesday: I decided to step away from MOTR in particular, and Python in my spare time in general, because I was having such a bad time with the saltate lugin.
Thursday: I started learning Godot.
Friday: I made a little more progress following along with the Godot tutorials.
Saturday: Nothing really happened apparently.
Sunday: I made a new category, for a project which is already several layers deep in weird detours.
Monday: I hopefully made some progress within the weird detours.

Next week, I'm going to try to get the Fair Fights stuff to the point where I can name a monster, finally.

Fair Fights 2023-05-29

2023-05-29T04:00:00-04:00

Okay, here we go. I've got a start on the roots, and they're ciphered. At this point, I need to kind of rate the resulting roots and pick out any patterns. As a run-up to this, I redid the part-of-speech classification for some of the roots. It maybe would have made more sense to do that before I picked out roots and ciphered them, but whatever, it's fine.

The current status is, some of the roots look a little hard to pronounce, and I don't really like some of them, while some of them I like a lot, and others show some patterns I could work with. I'm currently not up for rating and drawing connections for this stuff, but hopefully in the next few days.

Until then, I'm going to sleep.

Good night.

Fair Fights 2023-05-28

2023-05-28T04:00:00-04:00

So, Fair Fights. Here's the deal.

I've been messing around a bunch with the concept of a fakemon region. However, in the end, I decided to step away from "fan content for an existing franchise", and tried to come up with something similar but distinct. Playing a little bit of Cassette Beasts gave me the inspiration I needed to take the core combat mechanics in a different direction, and I ended up coming up with a few other ideas that further distinguished things. I won't go over those just yet, because I want to learn enough Godot to put together a prototype to determine whether all of these ideas are actually... good.

In any case, right now I want to focus on designing monsters, and part of that is coming up with names for them. As far as design goes, I want to consciously distance myself from Pokémon's design sensibilities, and to start with:

Instead of "balancing cool and cute", I want to come up with designs that are "both ethereal and powerful"
I want some other system for naming, different from puns, words smashed together, or just the name or sound of an animal

For the latter, I'm currently trying out "create a naming conlang", which feels like maybe a decision not made from an objective assessment of multiple alternatives, but oh well.

Here is my basic plan for attempting this:

Get a list of root words that seem relevant for coming up with names.
Add roots relevant to Fair Fights.
Pick a source language so I can be kind of lazy about coming up with roots.
Apply an Ostamer-style cipher to the source language so the derivations aren't blatantly obvious.
Look for patterns in the limited set of roots I'll choose, and try to regularize according to those patterns.
Come up with morphological derivation rules that take advantage of those patterns, as well as paradigms for conjugation and declension.
A few rounds of sound change.

All of these steps, except the last three, are either done or straightforward. I'm not sure about the third-from-the last or the second-from-the-last; we'll have to see. The last one could be a pain, because I'm not sure what the best way is to apply sound changes. I'm aware of a few different software packages where I'm not sure how to pick between them, and just doing everything manually is a possibility that I can keep in mind, but I kind of don't want to.

Anyway, I'll get to that once I'm done with the rest. Maybe it'll get me something I like, maybe it won't, but hopefully I'll be able to use what I learn in other projects. For now, I'm going to wind down.

Good night.

Diary 2023-05-27

2023-05-27T04:00:00-04:00

Um...

What did I do today? Spaced out a bunch, I guess.

I think the best use of my time would be to cheap out on this entry, and try to find a good angle for designing stuff in Fair Fight.

Good night.

Coding 2023-05-26

2023-05-26T04:00:00-04:00

A few quick thoughts on the Godot tutorials I've worked through so far:

Given my background, GDScript's syntax so far seems, like, aggressively approachable. We'll see if that keeps up.
I remember reading blog posts that had strong opinions about interactions between simulation steps and framerate. ... *websearch* ... Ah, it looks like I'm getting ahead of things.
The movement example feels painfully janky to luck at. I don't know if that's a result of running it in debug mode, having too many browser tabs open, or what, but I kind of expected my machine to be able to handle a simple scene. ... Okay, yeah, it was the browser tabs.

Hopefully, I won't run into too many issues in this area, since the game I want to make, codenamed Fair Fights (I need to find something with more unique initials) is turn-based-ish. Stutter won't open up weird exploits, it'll just look kind of bad, so hopefully I'll be able to avoid it, but we'll see.

Anyway, that's enough for now. I'm going to get to bed, and see what I'm up for tomorrow.

Good night.

Coding 2023-05-25

2023-05-25T04:00:00-04:00

All right, I'm still getting up to speed on Godot. It's late right now, but up next is actually learning about GDScript.

So, tomorrow, I'll hopefully be up for getting on with that. For now, I'll sketch out ideas for the game I want to try to make.

And cool off, because it is roasting in here, and my laptop isn't helping.

Good night.

Coding 2023-05-24

2023-05-24T04:00:00-04:00

Okay. I'm going to take a break from MOTR and MOTR auxiliaries for a while, so I'm currently looking at programming languages besides Python.

There are a bunch of languages that I'm varying levels of interested in, but I'm focusing on Godot, because one thing I want to try doing more is talking about this stuff with my wife, and that's simply easier if I'm focused on making games, or something like them.

Right now, I'm still going over the basics of the documentation.

...

And spacing out. Not going to get anything else done right now.

Good night.

Weekly Roundup 2023-05-23

2023-05-23T04:00:00-04:00

Wednesday: I gave some thought to extending the saltate plugin to handling Unions. Which is not a thing I need it to handle, but it seems reasonable.
Thursday: I implemented Union support, and wrote tests for a few other cases.
Friday: I noticed that the saltate plugin introduces implicit typecasts (as opposed to the explicit ones) if you do stuff that I don't care about, and tried to figure out what to do about that.
Saturday: I noticed another issue with the saltate plugin caused by inheritance. Absurd.
Sunday: Nothing worth writing about happened.
Monday: Nothing worth writing about happened.

Next week, maybe I'll work myself up to sort of finishing the saltate plugin, but also I'm thinking of messing with other programming languages than Python for a bit.

Diary 2023-05-22

2023-05-22T04:00:00-04:00

Okay, I think I had something wrong with me these last few days. I was kind of out of it, and then I felt much worse, and now, I'm... probably okay? We'll see.

I did mess around getting some stuff set up earlier today, and maybe I'll play with it later in the week.

For now, I just want to lie down and relax.

Good night.

Diary 2023-05-21

2023-05-21T04:00:00-04:00

I'm not feeling great today. Just... super out of it.

I've decided to deal with this by skimping on this post, and doodling ideas in my notebook.

Hm, yeah, okay, nothing else to say right now.

Good night.

Coding 2023-05-20

2023-05-20T04:00:00-04:00

Iiii got distracted.

However, I also noticed and fixed another issue in the saltate plugin. I'm not sure how that code is handling being such a bundle of issues...

Anyway, the problem was, I was erasing types too aggressively. Technically, this isn't a problem for my use case, because I barely use subclassing. But from a theoretical perspective...

Basically, if you have a class Base(Generic[T]) and a class Derived(Base[int]), then it can't be sound to treat a Derived as a Base[str]. Or any other example you care to come up with.

The solution is simple: for the subtype check (which I mostly have in place to deal with unions), erase the types on the inst arg, but not on the typ.

I still need to work out what I want it to do for non-final classes. I guess I can argue: "It might not be valid, but if it is, then the return value is of type typ, and any attribute on inst but not typ is, let's say required and of type Any, and if inst is non-final, maybe add a **kwargs: Any of some kind, except I'm not sure how to handle name collisions in that case..."

I'll think about it some more, but right now, I let things get way too late and I need to get to bed.

Good night.

Coding 2023-05-19

2023-05-19T04:00:00-04:00

Oh dear... I realized that the saltate plugin has some problems that the evolve code it's based on does not, and I'm not quite sure what to do about them.

Basically, the saltation operation is not valid if the runtime type is a strict subtype of the analyzed type, because attrs subclasses can add attributes.

That... sounds like a dealbreaker.

Not if I disallow passing subclassable types!

... Which of us is which, again?

Really, the only thing I'm not sure of is whether I want passing non-final types to be a hard error, or to simply complain.

But if it does fail, should I have it go all the way to Any, or accept some kind of best guess for what the arguments would be?

And I thought of more issues while I was pondering this stuff. Let's see which questions I need to answer...

What guarantees are broken if just the typ argument allows subclassing?
What guarantees are broken if the inst argument allows subclassing? Note that inst should be a subclass of typ, so it can't be the case that only inst allows subclassing?
Does the union-related code I copied out of the attrs plugin have the desired behavior in the context of saltate?

I'll have to think about this more later... For now, I need to wind down properly.

Good night.

Coding 2023-05-18

2023-05-18T04:00:00-04:00

All right. I've got support in for union types in saltate. I don't anticipate that I'll use it in MOTR, but it was going to bother me that it wasn't there. Let's see what's next to boost coverage...

Bounded typevar? Oh boy, that might actually break things? Huh, it did okay.

Well, there are a few more execution paths to puzzle out. I'll get those later. For now, I'm going to take things easy.

Good night.

Coding 2023-05-17

2023-05-17T04:00:00-04:00

Okay, let's think about the saltate plugin some more. There are a few transformations and checks I want to do. Currently, on line 131, it expects to receive a typ that is a TypeType, an AnyType, or just wrong. This needs to be extended to handling a UnionType. For a UnionType, we basically need to handle any of the items potentially being Any or otherwise not TypeType.

Something like:

return_permissive = False
for item in magic_func(...):
    item = get_proper_type(item)
    if isinstance(item, TypeType):
        pass
    elif isinstance(item, AnyType):
        return_permissive = True
    else:
        ctx.api.fail(...)
        return_permissive = True
        break
if return_permissive:
    return permissive_signature

I'm not feeling great about the exact structure there, but anyway. Line 136 converts the TypeType or the UnionType of TypeTypes into either the underlying type, or the union of the underlying types. I can get that by augmenting the above lines.

return_permissive = False
items = []
for item in magic_func(...):
    item = get_proper_type(item)
    if isinstance(item, TypeType):
        items.append(item.item)
    elif isinstance(item, AnyType):
        return_permissive = True
    else:
        ctx.api.fail(...)
        return_permissive = True
        break
if return_permissive:
    return permissive_signature

typ_type = UnionType.make_union(items)

The next code to handle is at 159. I'm not quite sure what has to be done here. There are two angles to take. The actual type comparisons have to reach into the unions, and convert all args to Instance types to Any. For diagnostics, I need to construct a sequence of names, and potentially do extra formatting to it.

I think I understand how all of this needs to be implemented, but I don't want to do it right now. At least I'm pretty sure I know what to do.

And what to do now, is get ready for bed.

Good night.

Weekly Roundup 2023-05-16

2023-05-16T04:00:00-04:00

Wednesday: I cooled off a little after getting the saltate plugin not quite working. While I was messing around, I discovered that the site theme was a little illegible, according to Color Science.
Thursday: I got the new theme mostly done, but elected to hold off due to a few issues.
Friday: I fixed a bunch of problems, and got the theme to a point where I'm happier with it now than I ever was with the old one, so I pushed it out.
Saturday: I tried to work out exactly how deficient the saltate plugin currently is. The answer is, just enough to bother me.
Sunday: We traveled.
Monday: We did not travel, but I didn't have anything to write about regardless, so eh.

Next week, I can either try to finish up the saltate plugin, or conclude that it's good enough for my current purposes, so I can push ahead using it. Maybe both?

Diary 2023-05-15

2023-05-15T04:00:00-04:00

I'm still taking things easy. Playing puzzle games a bunch.

Not coming up with anything substantive to put in a post.

I'm going to just cut this off early again, because otherwise I'd probably spend half an hour not making this post any longer, when I could simply have it not be any longer now.

Good night.

Diary 2023-05-14

2023-05-14T04:00:00-04:00

And we traveled today, so I'm not sure what to write about.

I'm just going to try to take things easy, maybe doodle some stuff in my notebook.

For now, this is it.

Good night.

Coding 2023-05-13

2023-05-13T04:00:00-04:00

Okay, here we are, looking at the saltate plugin again. The key thing here is, I need to understand exactly how the "proper types" in Mypy work, because there are a lot of them, and what I'm doing currently is not quite right. I need to work out how to activate each type of proper type, the correct behavior for it, and to devise corresponding tests.

Let's see...

TypeVarLikeType
- TypeVarType
- ParamSpecType
- TypeVarTupleType
- I believe these should be treated as either their upper bound, or Any. Maybe fail?
UnboundType I'm not sure what to do with this if it shows up.
CallableArgument Should not show up.
TypeList Should not show up.
UnpackType Should not show up.
AnyType Should already be handled.
UninhabitedType Should be treated as Any. Maybe can obtain using NoReturn?
NoneType It's None. Should fail.
ErasedType Not sure how to hit this.
DeletedType I don't have an intuition for how this should interact with any of the plugin. Should probably just fail.
Instance I'm trying to handle this already.
FunctionLike: CallableType and Overloaded These should fail.
Parameters Probably should not show up.
TupleType Should fail.
TypedDictType Should fail.
RawExpressionType Should not show up.
LiteralType Should fail.
UnionType Needs special handling. Every argument of a union needs to be transformed recursively.
PartialType If it shows up, the correct action is... probably complicated? Like okay, from a "do the types match" perspective, the correct action is to replace the placeholders with Any. When it comes to the typ argument, this shouldn't show up, so as inst, when the types match, we need something like an attrs class with a field of type T | None. I think that makes a PartialType.
EllipsisType Should not show up.
TypeType Must be handled, with specific cases for Instance, AnyType, TypeVarType, TupleType, NoneType, CallableType.
PlaceholderType Should not show up?

So, to summarize:

typ argument must handle TypeType and UnionType of TypeType. Within the TypeType, it needs special handling for AnyType and Instance. Maybe convert TypeVarType to its upper bound, but probably don't. Actually, there's even more subtlety here, bleh. I kind of hope a constrained typevar just gets checked with the concrete values so I don't have to think about it? Everything else should fail and coerce to Any.

inst argument needs to handle the same types as typ, and maybe also PartialType?

From this, we see that the plugin is mostly done, but it still needs work, mostly to cover the union cases. I'll get to that whenever. I'll at least write a test to see how messed up it all is.

Okay, it's definitely messed up, and I'm not currently up for fixing it, so, whenever. For now...

Good night.

Diary 2023-05-12

2023-05-12T04:00:00-04:00

Well, here we are. I'm posting this entry, and redoing all of the pages on this site, because I've finished updating the theme files. Now, they're Selenized. While I was at it, I redid the light/dark mode theming to provide a proper fallback on low-end browsers what don't show up on Can I use. (I mean, I guess it shows up better on Internet Explorer. (And Opera Mini.) (I have not attempted to confirm this.)) And, lastly

At last!

:) (glitchy)

Somewhat improved legibility!

:) (glitchier)

The old version of this icon was too spicy to actually be used!

Yep, we've come so far. Maybe I'll remember to actually use that later.

Anyway, at some point, I need to fix some bugs I discovered by rotating the saltate plugin in my head, but for now, I should take things easy. I'm going to be worrying enough about making sure there aren't any show-stoppers in this post.

Good night.

Diary 2023-05-11

2023-05-11T04:00:00-04:00

All right. I've made a bunch of progress updating the color scheme for the blog. It's not quite ready to go; I need to tweak a few CSS rules before I can expect it to work right.

It's a little obnoxious that I can get so far basically on autopilot, and when it comes to a few finishing touches, those are seemingly going to take more effort than the rest of the work put together.

...

I just now hopefully got the CSS side of things right, and now what I need is to validate and remake some assets. I think I'll just cut this entry here and take things easy.

Good night.

Diary 2023-05-10

2023-05-10T04:00:00-04:00

I really went hard polishing the new plugin, so I'd like to take a bit of time to cool off. I'll be trying to think about writing, but one other thing that I'm going to be thinking about is trying to make my blog more readable. If you run ANDI on this blog, some things are okay, but it's not happy with the color scheme. (And there are other issues, but it's really unhappy with the color scheme.)

I've got some ideas in mind to deal with that, but the first thing I thought of is not a drop-in replacement, so this may end up being some kind of a fun side project.

Anyway, it's a little late and I don't want to force myself to go further, so I'm going to call things here. (Maybe I should look into getting back into longer-form posts sometime...)

Good night.

Weekly Roundup 2023-05-09

2023-05-09T04:00:00-04:00

Wednesday: I looked over how the attrs plugin handles the evolve function.
Thursday: I put together a weird implementation for saltate.
Friday: I upgraded Mypy, and had trouble distringuishing between Tuple and tuple in the failure messages.
Saturday: As I spun up the plugin development, I realized that I'm going to want to pull this out into a separate package as soon as I can.
Sunday: I started writing tests, and using their failure details to refine the plugin design.
Monday: I got the broad details of the plugin pretty much locked in, so it's in a good shape now.

Next week, I'm going to try to fully polish the plugin, as well as develop the generic motrfile enough to pull the plugins out into their own packages.

Coding 2023-05-08

2023-05-08T04:00:00-04:00

I spent a lot of today smacking my head into Mypy plugin stuff, and I think things are coming together and making more sense to me. I've got better error messages and the signatures being returned are in better shape.

From all of this, I now understand that I'm missing a specific case:

saltate(typ: type[T], inst: Any, **changes: Any), when T is an attrs class, should get typed like: saltate(typ: type[T], inst: Any, **changs: <the attributes of T>) -> T

So, let's see if I can untangle that from the current plugin code.

...

Okay, that wasn't too bad.

I basically had to split apart a fancy conditional, and move a line up in the function, so I could construct a mediumly-permissive signature. The result will sometimes construct somewhat peculiar signatures, but those edge cases should be accompanied by a type checking failure, so I don't believe there's a serious problem with that.

There are a few big misses remaining in the plugin code that will be filled in by the positive test cases, but I want to focus on the last thing I don't quite understand:

Is the length of ctx.args determined solely by the function declaration, or does it somehow depend on the call being processed by the get_function_signature_hook?

If it's the former, I don't understand how I'd write tests for that branch. If it's the latter, I still haven't figured out how to go down that branch. I'm going to take the slightly spicy option of just asserting that my library method will always have exactly three arguments, from the perspective of names available within it.

Anyway, once I have coverage on the hook function locked down, I'm going to work on expanding coverage in the helper functions. This is likely to result in commenting out some blocks or something, but we'll see how that all goes when I take the time to do this stuff. I'm also going to have to comment all of this code thoroughly.

But right now, I'm going to wind down.

Good night.

Coding 2023-05-07

2023-05-07T04:00:00-04:00

Okay. I've got tests for the saltate plugin. They're not passing yet, because I haven't nailed down the desired behavior. In order to get a proper idea of the desired behavior, I'm coming up with failing test cases, and inspecting the resulting failure messages.

Careful inspection of the error messages and the coverage data has allowed me to conclude...

That I still don't understand how Mypy represents types.

Anyway, I ended up reimplementing parts of the attrs plugin. At the rate this is going, I might accidentally make the plugin compatible with older versions of Mypy. I did the reimplementation because I was pulling out helper functions from the attrs plugin, and they really didn't expect to be used with anything that wasn't evolve. I'm still pulling out helper functions, just, less so.

Let's take a look at the improvements I want to make:

Failures generated by the reimplemented helper functions are associated with the function, rather than the specific argument.
Failures associated with the typ argument use the wrong type name.
Once the above fixes are made, I want to use a more permissive signature for the relevant returns, because then the plugin-generated type errors are handling everything that is needed.

I don't know if the first one is possible. I believe I handled the second one, because the attrs plugin code is flexible, and I just needed to change which arguments I'm passing.

I worry that the second one will result in more deep-dives into the Mypy source code, because I have to somehow go from "a function call context" to "the context of an argument to that function". That sounds like something that someone would have needed at some point, but I really don't know, and I don't know what it would take to hack it together if people have gotten by without that somehow. Hm. I'm going to try retrieving the expression node from the function argument context, and...

There. We. Go.

I've got a few more tweaks I want to make to presentation, then I can get to work making tests of the updated parts of the plugin pass.

(I should maybe make sure that the positive tests pass, now that the negative tests are somewhat locked down...) Anyway, it's late, I should wrap up.

Good night.

Coding 2023-05-06

2023-05-06T04:00:00-04:00

Last night, I figured out why updating Mypy in my test environment made some tests fail: the capitalization of some types changed. This might have been easier to notice if the change markers were meaningfully aligned with anything, not sure what's up with that.

In any case, with that done, I have what I need to plan out the tests. For that, I have a rough idea of the kinds of types and situations I need to handle, and given those, I just need to fill it all in.

However, I expect that at some point, the test times will end up being so painful that I'll end up switching focus back to the generic motrfile repo, so I can pull out the plugin-bearing code ASAP.

Anyway, maybe I'll get on writing the tests after this, maybe I'll do it tomorrow. Either way, I'm going to wrap up now.

Good night.

Diary 2023-05-05

2023-05-05T04:00:00-04:00

Well, I'm in better shape than the last diary post. I got carried away with a distraction, but I didn't, like, injure myself on it.

I'm going to not try to "make up lost time", and just let things go for the rest of tonight. Maybe take some notes on paper for the saltate plugin testing. I'll at least update the requirements file that's relevant there...

Aw heck. That more than doubles motr -t nocov execution time, and makes the type tests fail for made-up looking reasons.

So, here's how things go:

Tonight, sketch out the tests I'm going to have to write.
Later, hope that I can troubleshoot the existing test failures.
In general, anticipate the release of Mypy 1.4.0.

For now, I have to get away from my screen.

Good night.

Coding 2023-05-04

2023-05-04T04:00:00-04:00

Well, I bashed out code for the saltate plugin last night, but it turns out that trying to make the actual implementation have mildly reasonable annotations causes Mypy to raise a bunch of sensible objections that I don't really know what to do about. I could loosen the annotations and try to put more type-checking smarts on the plugin, but that would mean that nearly anything would typecheck without the plugin.

One of the issues is, I'd really like to be able to bound a TypeVar as "any attrs class", and according to my intuition and a cursory Google search, that's not a thing.

Let's review:

typ: type[T]: hmm
inst: U: hmm
**changes: Any: no notes
-> T: I mean, that makes sense

I don't think there's any sensible way to express the desired relationship between T and U. If I could just express "Hey, this is some attrs class..."

Actually... If I could just forbid it from typechecking without the plugin rewriting the signature? Hm.

So, like, if I created a typing-only attrs class? This is a silly idea; I have to try it.

...

That

worked

???

It all typechecks, despite clearly being objectively ludicrous. "Here, reinstantiate this class that doesn't exist with arguments that it wouldn't take if it did."

Well, next up is seeing whether the plugin actually works. I don't have the stamina to write the tests currently, and I also need to update the requirements file for testing, which is probably going to slow it down even more.

But, anyway, that's one obstacle knocked down.

We'll see when I'm up for more. For now, I'm going to wind down and get to bed.

Good night.

Coding 2023-05-03

2023-05-03T04:00:00-04:00

I don't feel like touching MOTR directly right now, so let's take a look at how Mypy supports attrs.evolve.

It expects two args (inst and **changes?). Then it mostly focuses on args[0][0]. Is Mypy's usage of nested lists documented anywhere? I don't remember understanding it with the last plugin I wrote, just figuring out what made the test cases pass.

Anyway, it gets the type of inst, and operates on that. There's a helper function I'm going to have to get access to, which determines the types of the fields. Once the fields are gotten, it does a bunch of stuff that's pretty straightforward for evolve, and will be less so for saltate.

Let's see what's different.

saltate has two required positional arguments, typ and inst.
It needs to determine the type of typ. That is, given "this expression is of type type[T]", it needs to get the representation of T.
It needs to confirm that the base types of T and inst are the same; this should not fail if they aren't generic, because I'm pretty sure there's some way to make that case matter, even if it's not occurring to me just yet.
It needs to go through the overall flow for both types, eventually generating two different lists of fields.
The one from typ should be used, but it needs to be compared against the inst list, to determine which fields should be required instead of optional. Maybe this can be done with simple equality, or maybe I need to try to unify the types.

In any case, let's work through this in order.

Probably look at ctx.args[0][0] and ctx.args[1][0], after checking that len(ctx.args) == 3, len(ctx.args[0]) == 1 and len(ctx.args[1]) == 1.
typ should be a TypeType? The type we want is the item field?
If both types are Instance, then compare by the type field, otherwise compare normally? If the comparison fails, do no further processing, otherwise proceed.

That gets us to fields, and the check we need to make is whether a given inst field is a subtype of the corresponding typ field. Ah, that's what mypy.subtypes.is_subtype is for.

I think that's all the information I need to write the hook for saltate. I'm going to hold off on writing it, because I still want to figure out stuff like where to put it. Judging by how I positioned other general-purpose stuff, it looks like it should go in a module right under the motr package. (At least, until and if it gets its own package.)

Let's see if I'm up for working on this properly tomorrow.

Good night.

Weekly Roundup 2023-05-02

2023-05-02T04:00:00-04:00

Wednesday: I started thinking about the saltate concept, and worked on a workaround for a Mypy... I think it's confirmed as a bug, that I found.
Thursday: I filed the bug, and started working on wrapping pyinstrument.
Friday: I refined and updated the Placeholder concept.
Saturday: My efforts to get Placeholders working resulted in rewrites in other parts of the code.
Sunday: I wrote tests for the pyinstrument wrapper, and immediately became suspicious of my test primitives.
Monday: I looked over the diagnostic methods on Compendium, concluded that I only need one more, and wrote it.

Next week, I'm going to think about how to improve these tests, but also start taking notes on what's needed for the saltate plugin.

Coding 2023-05-01

2023-05-01T04:00:00-04:00

So, I was focused on other stuff and generally kind of out of it today, but let's think a little about how I want to do Compendium diagnostics. Well, these ultimately focus on private fields of the Compendium class, so I guess they have to be methods on it?

It's late and I have a few other things to do, so let's just take a quick look at what's in here that I need to deal with.

First off, let's switch to the future topic...

And what we have currently:

"Which Tasks have no outputs?"
"Which Tasks are not transitively required by a named output?"

Stuff that's used for other reasons:

"What are the requirements for a Task?"
"What outputs are referred to by a given name?"
"What Task corresponds to a given output?"
"Which names are selected by default, and which are not?"

I additionally want:

"Provide all Tasks in the compendium, either as an iterator or a container."

Beyond that, I'm not really sure.

Well, if I can't think of anything else, then I'll just... do it.

...

And, it's done. That was one of the shorter ellipses.

Anyway, I'm tired and I'd like to wind down, so I'm going to wrap up now, and figure out how to actually use all of this stuff later.

Good night.

Coding 2023-04-30

2023-04-30T04:00:00-04:00

I've very quickly written "new" tests for the code I've added, and that all seems to work and hit the proper parts of the code. However.

Looking at the tests, I see that they're very superficial. There are very basic questions that I should be asking about the output of this code that I'm currently not sure how to ask. I remember considering some of them in the past, so I'd better take a quick look at the relevant module...

So, the Compendium, at least in this pre-future version where that's not actually what it's called, defines two diagnostic methods. These methods help to track down basic issues with the Compendium, and I should use them outside of the apparently just one test that they're currently in.

But I also need diagnostic stuff like: provide every command where the first argument ends with the given name. From there, I can check over the remaining arguments. This will let me confirm stuff like "commands will have everything installed required to actually run". Tomorrow or later, I'll try to work out what kind of interface I want on that. For now, I'm going to wind down again.

Good night.

Coding 2023-04-29

2023-04-29T04:00:00-04:00

I think I understand where I went wrong with some of the design I've been working with, but I want to make sure I don't cause myself even more problems trying to deal with it.

Basically, right now, I have a class called command.Metadata, which is parametric over installer types, and I think I actually need to make it just accept any installer type. It's already supposed to have the groundwork for this in there, sort of. Now, what this is going to mean is that I need to redo the placeholder filling-in. Here's how that looks, I think:

Two different functions for filling in an executable. One is strictly typed, and operates unconditionally. This is what is currently used everywhere. The other one needs to be loosely typed, and to take an additional parameter to condition whether to replace or not. Basically, if that parameter matches what's in the executable exactly, then do the replacement. Otherwise, leave it be.
The command.Metadata class (which probably needs a new name) is no longer parametric.

...

Okay, I did a bunch more work, some of it focused around undoing stuff I did before. There's more work where that come from, and I need to think about some things.

Like, now that I'm just doing replacements at the level of one PathWith, should I be using combine at the... Wait, I'm already doing what I was considering doing. Okay, this code is a little bit of a rat's nest.

...

Or.

Or I could get distracted and just do it.

At the very least, the various checks all pass, except for the one that I didn't expect to.

I'll leave things like actually testing this new code for later. For now, I'm going to be glad that I seem to have pulled this together, and hopefully improved the design of this part of the code.

Good night.

Coding 2023-04-28

2023-04-28T04:00:00-04:00

Okay, I'm trying to reconsider how I'm handling the passing of requirements relevant to a command that needs a module argument passed. I'm getting a sense that there's something a little iffy in how all of these components fit together, but I think I've figured out what I need to get unstuck for now.

Basically, I need to make it possible to instantiate and combine placeholder installers, but not to resolve with them. Specifically, a placeholder should contain an instance of the corresponding installer type. Then, the _command.Metadata needs to retrieve the nested installer under the placeholder label, and merge it with the value under the new key, if any.

After that, my plan is to rework the helper code. The point of the helper code is to define builder classes that get out of the way as soon as possible, by constructing instances of common classes. There are two things that need to be done. One is to convert a property to a method, and change a field on the class to an argument to that method. The other is to add a distinct method that takes different argument types.

Let's try doing all of this, and see where I get tripped up.

...

Hm. Looks like I need some way of knowing which labels are getting filled in.

...

Okay, so this function implementation is not clean, per se, because I'm running into some issues where some part of my thought process wanted to pretend I had dependent types, and that's simply not the case...

...

Okay, that's part of it done. Let's make sure I didn't introduce any new typing issues. ... I have introduced a new typing issue. But I dealt with it. I just needed to make the updated function even grodier.

Anyway, now I just have to... Hm. Hm. I'm going to need to think about this some more. There's a helper method that assumes it's getting an executable, but if I just change how that helper is put together a little...

This should produce so many typing issues...

I'm paring down the typing issues, but I don't feel like putting in the work to deal with this last set right now. Tomorrow, or over the weekend maybe. And once I get this stuff to pass, I need to make a thorough check of whether the helper functions are actually helping.

Actually, I can see one thing I'd like to have: a map method on PathWith. That should alleviate a bunch of heartburn with these changes.

Okay, that was quick to write. I'll make use of it later, and finish up the changes to the helpers.

For now, I really should wind down.

Good night.

Coding 2023-04-27

2023-04-27T04:00:00-04:00

With some fiddling, I managed to both get Mypy to accept my code for now, and to produce an actually decently-small example of what's going on to file a bug report. It looks like that's getting looked at.

Anyway, let's take a look at some of the motrfile code I want to make an equivalent of in the new system.

yield from run_pytest(
    "profile",
    "profile",
    (
        "pyinstrument",
        "--renderer",
        "html",
        "--outfile",
        profile_dir.map(INDEX).as_output("profile"),
        "-m",
        "pytest",
    ),
    "pytest-profile",
    (profile_dir,),
)

I... hm. Just so we're clear, these functions that mostly take string arguments... kind of suck? Anyway, let's break it down.

The stuff I care about the most is the tuple in the middle.

pyinstrument the executable
--renderer html or --renderer=html is needed for output; other values are possible, but I'll leave it at this for the initial release
--outfile <path/to/file> is the other part of output; the existing new-style code should handle the profile_dir input stuff later on
-m module runner argument; this is proven out with the Python wrapper

I fiddled around, and got everything to a point that seems to make sense... and then it failed type checking because I hadn't accounted for placeholders in some of the utility code I'm trying to use. (Side note from looking at the names I'm using: wow, some of these names are bad. Like, there's a "Labeled" type that doesn't actually require a label???) Anyway, placeholders. In a fancier system, I would be lobbying for higher-kinded-typevars or something like that about now, but there's no point when there's only one instance to work with currently.

...

Hm. I appear to have dug myself deeper into a hole here. This helper code isn't really taking to the placeholders, and I'm going to need to think some more about how to address this, and avoid making any rash decisions. (I mean, I could make some, like, alternative methods to handle placeholder versions, but that would... sort of suck.)

I'm going to take a few days to ponder this, and try to wind down for now.

Good night.

Coding 2023-04-26

2023-04-26T04:00:00-04:00

Okay, here we go... I've got a bit to catch up on.

I'm leaning towards closing my feature request, but I kind of want to see what other people think.
I figured out how to write the plugin to catch problematic class definitions. It's a pretty gross implementation, though, so I'm looking into an alternative.
If I'm writing a plugin anyway, the discussion on the feature request made me realize what I'd want a plugin to do.

Let's get right into it.

Code:

def saltate(typ: type[T], inst: Any, **changes: Any) -> T:
    return attrs.evolve(inst, **changes)

Plugin: ... like the new attrs plugin code, but unfortunately, more complicated. Basically, it needs to do all of the legwork of the builtin plugin, but also check that the underlying types of typ and inst "match", but need not unify, derive all arguments from typ, and mark any that are a different type from the attributes on inst as mandatory.

The basic concept here is that, because what I want is a kind of "discontinuous evolution", it's "saltation", and it needs to provide a specific target, as if it's a cast. This addresses some of the problems uncovered in the feature request discussion, and also hopefully makes it obvious that, hey, if you don't need to specify a type, probably just use evolve.

Unfortunately, it is too late at night for me to puzzle out plugin code right now, so I'm going to go over what else I did today.

After getting a bunch of helper functions written earlier, I finally put it all together, and moved code out of the python module. At this point, it should be safe to create profiling wrappers.
In order to have things work properly with the plugin I want to write, I finally unpinned Mypy from pre-1.0, addressed the type issues, one potentially valid and one that seems incredibly made-up to me, and then pinned to master, because otherwise I'd just have to pin to 1.1.*. I've got to say, I really hope I get Mypy yelling at me about unused type ignores from this, because I can't figure out which case I'm supposed to be hitting.

However, the more I think about it, the more I think "No, there has to be a way to make Mypy make sense here." So I guess I'm going to be bouncing off of that for a bit.

...

Okay, that's weird. I managed to get rid of one instance of the error by just annotating stuff until Mypy admitted there wasn't actually a problem, but it's not working a second time.

Okay, that's... not the greatest solution, but I can't find fault with it from a simple inspection perspective.

facts: _requirements.Requirements[_cmd.EnvVar[_cmd.CmdArg]]
facts = value.map_facts(map_func)
facts = typing.cast(
    "_requirements.Requirements[_cmd.EnvVar[_cmd.CmdArg]]", facts
)

"What does the cast accomplish?"

It makes type-checking pass.

Anyway, now I've got a clean basis for working on saltate, but before that, I should probably try to extend the code some. It's possible I've forgotten some more improvements that I want to make. But for now, it's late.

Good night.

Weekly Roundup 2023-04-25

2023-04-25T04:00:00-04:00

Wednesday: I planned extensions to the high-level MOTR API.
Thursday: I decided to take a detour and try to unpin Mypy. This resulted in filing a bug.
Friday: Well, Mypy is fixed, the fix just isn't released yet. So I updated some of my Trio usage, and it went much smoother.
Saturday: I played a single game of Vampire Survivors for almost two hours. Do not play a single game of Vampire Survivors for almost two hours.
Sunday: I decided to try to address an issue with how I'm using attrs.
Monday: I ended up trying a few different tacks, and not really being satisfied with any of them yet.

Next week, I'm going to spend a little more time trying to make some kind of plugin to detect what I want to detect, but so far it's not going great.

(I tried out one of those LLMs that's supposed to code for you. Absolutely uncanny simulation of a sleazy used-car salesman who's been abducted, and is being forced at gunpoint to produce code samples.)

Besides that, I'll be working on the actual updates.

Coding 2023-04-24

2023-04-24T04:00:00-04:00

So, I ended up tabling the "remove defaults" idea in favor of putting in a feature request to Mypy to see where that goes. I can anticipate some potential objections to it, and I'm not sure how to address them, so I need people who understand Mypy better than I do to discuss it some.

There are two major reasons I swung around to a feature request:

What I was trying to accomplish with the code changes was to make a worse version of attrs.evolve.
I couldn't figure out a good way to get a linting framework to do what I wanted.
- Semgrep doesn't seem to support regex operators in token names, so if I want to account for all of the different reasons that the rule might or might not apply, I'd need a combinatorial explosion of rules. Also, I couldn't find a way to make it detect "a type expression that uses any typevar".
- Although Pylint gets me as far as detecting attrs classes, it's not obvious to me whether the information I actually want is present in the AST.
- I honestly didn't even try with flake8.

My guess is that, with some effort, I could get Pylint working nicely with this idea, but I'd rather focus on the feature request for now.

Let's see about chasing down the information that got me down this tangent in the first place...

I want to put a Placeholder into an Executable, and I have a means of taking it back out, given the proper data.
At the other end of the pipeline is the ParametricCommandMeta, which needs to carry out updates to a CommandBuilder.
The CommandBuilder needs the ability to update an Executable, which we have, and a command.Metadata, which we don't. Yet.
The Metadata is the most troublesome to update, because it requires conditionally mapping the Executable update over three different data structures.

In terms of raw volume of code, it's not clear that doing this would constitute a win, but it would mean moving code out of the more data-y modules, so I at least want to evaluate how much of a mess it would make.

Let's treat the top-level data structures as mostly straightforward, and focus on the type fiddling.

Args is one of the baselines, because it just has a MaybeExecutable.
EnvVars looks a little scarier due to the use of EnvVar, but I can just map a lambda over that, no problem.
IOSet is mainly confusing because it uses a more restrictive set of types in the left-hand side of the union.

Now, the thing that hopefully makes this less confusing than the existing functions is, I don't need special handling for different entry point types. Everything just gets passed through, no questions asked.

Before I mess around with new code, I want to investigate using the techniques from this comment in the existing code.

(Update: Things aren't currently looking great for the Mypy feature request, so I'm trying to nail down what I'd want out of a plugin. Like, the brief is, the constructor for a generic attrs class shouldn't have any optional arguments, and the question is just how to express that in terms of whatever I use to lint.)

Anyway, back to messing with the code...

There's some nested code that doesn't need updating because it doesn't need overload.
There's some code that can't work with this because I need to key off of the type parameters, but fortunately, this isn't an all-or-nothing thing.

So, I've proved that this still works, and the behavior is preserved (up to and including the mystery errors from recent Mypy versions).

At the moment, I just want to wrap up and get to bed, but this was nice for proving out the approach for handling the placeholder updates. I'll try to do that in the next few days, and hopefully it'll get me some data points for those mystery errors.

Good night.

Coding 2023-04-23

2023-04-23T04:00:00-04:00

I did a little work on MOTR after last night's entry, and I ended up with an idea that I'm not sure whether I should believe:

"Generic attributes should not have a default value in the base constructor, and the class can provide alternative constructors that do provide default values."

I may even consider a strong statement, starting instead with "Generic classes should not have default values for any attribute".

I'm pondering this, because I can't see a way to make evolve handle changing the parameters to the type; this means that "modified" values must be instantiated from scratch, and that means that, if I add another field to the class definition, it will be silently dropped from the update if it has a default value.

This has implications for:

io.Output
python_helpers.PythonCmd
artifact.Input
artifact.LabeledConverter, technically, I guess.
artifact.BasicOutputConverter
command_builder.CommandBuilder
parametric_command.ParametricCommandMeta
parametric.Parametric
command.Metadata

Hm. Let's see how much it hurts if I start changing these...

So far, it hasn't been a big deal, but this seems fiddly enough to remember and to deal with that I'm now looking into semgrep, or maybe something similar, to check the codebase for a highly specific pattern. Like, "when a class has one of these decorators, and inherits from an expression involving instances of this type, make sure that any values set at the class level either have a specific type annotation, or the value is a call to a function that does not pass some specific arguments". Sure.

It's a bit late to write that right now, but I'll get on it tomorrow. For now, I want to wind down.

Good night.

Diary 2023-04-22

2023-04-22T04:00:00-04:00

Me several hours ago: "I wonder what it's like to play Vampire Survivors on Endless mode"

Me now: "I wholeheartedly regret that decision"

I would have given up much sooner if I hadn't been switching off hands to control movement as my wrists wore out...

Point is, my brain is jangly now, and I can't really do much tonight as a result.

Good night.

Coding 2023-04-21

2023-04-21T04:00:00-04:00

People are working on fixing Mypy, so I'll see improvements there... after at least two more minor versions. I'm going to put off the unpin until then, and maybe look into putting together something for the other errors I hit, just so I can pin down whether the changes are intended or not.

Aside from that, I got prepped for some future release of Trio, so that's pretty great. I'm not sure precisely how valid my code to handle ExceptionGroups is, but the tests pass, at least for now, so eh.

Anyway, now that I've fully characterized the parameters of my tactical retreat from messing with Mypy, I'm all set to investigate the right places to define and use Placeholder next time I touch the code.

Good night.

Coding 2023-04-20

2023-04-20T04:00:00-04:00

I'm going to try and get Mypy unpinned. Let's see what kind of shape it's in...

(motr-dev) ↪ hg diff
diff --git a/requirements/mypy.txt b/requirements/mypy.txt
--- a/requirements/mypy.txt
+++ b/requirements/mypy.txt
@@ -1,3 +1,3 @@
 lxml
-mypy<1 # My weird code.
+mypy
 trio-typing

And when I run motr -t mypy, it

src/motr/_api/cli_types/parametric.py:512: error: Argument "instantiate" to "evolve" of "Parametric[T]" has incompatible type "Callable[[PerItemParametricMapping[motr._api.cli_types.box._Unlabel], PerItemParametricMapping[motr._api.cli_types.selection._Unlabel]], Generator[Union[Action, ActionInput, ActionOutput, TargetName, SkippedName], None, T]]"; expected "Callable[[PerItemParametricMapping[motr._api.cli_types.box._Unlabel], PerItemParametricMapping[motr._api.cli_types.selection._Unlabel]], Generator[Union[Action, ActionInput, ActionOutput, TargetName, SkippedName], None, T_co]]" [arg-type]

I mean, I think that sounds straight-up incorrect, but

Fiiiiine.

So, about two weeks ago, Mypy got another release, and I got more errors. With some experimentation on a reproduction that doesn't recite the entire text of Moby Dick, I've determined that it's probably fine if the following two conditions are met:

The call to evolve() is in a method on the relevant class.
The typevars associated with self exactly match the ones on the class line??? That is, from my testing, it appears that you can't annotate a method on a class defined with class MyAwesomeClass(Generic[T1]): like def my_awesome_method(self: MyAwesomeClass[T2]) -> T2: even if T1 and T2 are identical except for the name???

There is no way that second one is intended behavior. Ludicrous. The first one is a little obnoxious, but I'm not confident that that wasn't intended? Anyway, let's see what I need to handle this...

Okay, wait. Hold on. It instantiated the constraints on a typevar, then failed to unify the typevar with the constraints? Okay.

Okay.

I've got to file this, because that is actually impossible to satisfy.

So, I guess I can consider trying to move the pin out, but I'm frankly not impressed by how Mypy 1+ has handled MOTR's code. Kind of weird how the scrunkly plugin stuff hasn't posed any obvious problems, but whatever.

Anyway, getting all het up about how messed up the new evolve() handling is ate up the night, so I guess I'm not doing this. And I'm kind of skeptical about whether it's worth trying to get to 1.1+.

I'm going to wrap things up for now, and see what I'm up for later.

Good night.

Coding 2023-04-19

2023-04-19T04:00:00-04:00

Okay, here we go.

I want to create a placeholder value for installer labels to allow me to convert a bunch of a function I've written into a combination of a utility function and some static data, so I can define different data in other contexts.

Here is a first shot...

@attrs.frozen
class Placeholder(Generic[_installer.TArgs]):

    uninstantiable: NoReturn

    def resolve(self, __environment: object, __cmd: object) -> Facts[Input[Path]]:
        raise NotImplementedError

    def combine(
        self, __other: Placeholder[_installer.TArgs]
    ) -> Placeholder[_installer.TArgs]:
        raise NotImplementedError

The basic idea is that there's some way to take a ParametricCommandMeta[TEntry, TProgram, Placeholder[_installer.TArgs]], pass it a ParametricCommand[Module, Any, _installer.TArgs], and synthesize them into a ParametricCommand[TEntry, TProgram, _installer.TArgs].

I'm pretty sure I can write the code to do that, no big deal. I'm just not sure if it's a generally valid transformation, outside of the context where I want it.

All the same, it seems like a better idea to just go for it.

Now, how to lay this out. I'm leaning towards, actual definition in parametric_command, but re-export in the installers namespace. Some amount of support code to handle the type twiddles will probably be necessary, but I'm going to make a bold prediction that it won't be a big deal.

I'll get started on this tomorrow, because right now, I need to wind down.

Good night.

Weekly Roundup 2023-04-18

2023-04-18T04:00:00-04:00

Wednesday: I wrote a bunch of things that weren't a blog entry, thereby not leaving me the time to write a blog entry.
Thursday: I considered some changes to the layout of MOTR, including a few things that turned out not to matter.
Friday: I continued to consider the changes, and came to a decision.
Saturday: I tried to trace the flow of MapOverSelection through parametric_command and artifact, which was... still obnoxious.
Sunday: I made all of the planned changes.
Monday: I backed out some of the changes in order to keep things usable, but I did keep the major improvements.

Next week, I'm going to try to remember what I wanted to work on next. I think I've managed to figure out some of it...

Coding 2023-04-17

2023-04-17T04:00:00-04:00

Pro: The tests pass now.

Con: They were coupled to the old design, not in the sense of "They're doing things the old way, and need to be switched to the new way.", but in the sense of "There is no new way, so I need to walk back a bunch of my changes."

Pro: I didn't have to walk them all back, and I was able to keep a bunch of the major improvements.

Con: This merge is going to suck.

Let's see how that goes...

(motr-dev) ↪ hg up 39-documentation
switching to topic 39-documentation
56 files updated, 0 files merged, 20 files removed, 0 files unresolved
maxchase at maxchase-ThinkPad-P1-Gen-5 in ~/D/s/m/motr
(motr-dev) ↪ hg merge 87-test-coverage
file 'src/motr/_api/cli_types/map_over_selection.py' was deleted in other [merge rev] but was modified in local [working copy].
You can use (c)hanged version, (d)elete, or leave (u)nresolved.
What do you want to do? d
merging src/motr/_api/cli_types/artifact.py
merging src/motr/_api/cli_types/command.py
merging src/motr/_api/cli_types/installer.py
merging src/motr/_api/cli_types/parametric.py
merging src/motr/_api/cli_types/parametric_command.py
warning: conflicts while merging src/motr/_api/cli_types/parametric_command.py! (edit, then use 'hg resolve --mark')
merging src/motr/_api/package.py
warning: conflicts while merging src/motr/_api/package.py! (edit, then use 'hg resolve --mark')
merging src/motr/_api/tasks/cmd.py and src/motr/_api/actions/cmd.py to src/motr/_api/tasks/cmd.py
warning: conflicts while merging src/motr/_api/tasks/cmd.py! (edit, then use 'hg resolve --mark')
merging src/motr/_api/tasks/io.py and src/motr/_api/actions/io.py to src/motr/_api/tasks/io.py
warning: conflicts while merging src/motr/_api/tasks/io.py! (edit, then use 'hg resolve --mark')
0 files updated, 4 files merged, 2 files removed, 4 files unresolved
use 'hg resolve' to retry unresolved file merges or 'hg merge --abort' to abandon

... Well, that probably could have gone worse?

...

That wasn't so bad.

I haven't made all of the changes that I could make, but I'm going to hold off on taking it further for now. The thing to focus on after now, is to refresh my memory about what is a problem for the remaining wrapper code, because I've completely forgotten what was supposed to be problematic with this code.

I'll start on that in a few days. For now, I need to wind down.

Good night.

Coding 2023-04-16

2023-04-16T04:00:00-04:00

All right. I ended up just jumping in and updating code until Mypy stopped complaining. This resulted in something happening with the tests, and I'll try and get them passing again later.

Basically, I just did what seemed to make sense, and nothing obviously broke yet. I should be able to delete a lot of code now, but I don't yet know for sure.

We traveled again today, and I feel like I was lucky to get this much done, and I don't have the energy to write this up in any more detail.

Good night.

Coding 2023-04-15

2023-04-15T04:00:00-04:00

Okay, so, the next thing to work on is updating parametric_command to use the new class I threw together. I could try working methodically through the file, or making changes at random until Mypy stops yelling at me.

Or, I could try simulating that, but describing the code changes instead of trying to actually make them all at once.

Let's start with the definition of Part. Part now has to involve _parametric.AdaptiveParametric[BlahBlahBlah]. This requires a minor and obvious change to the _optional_parametric helper. In addition...
There is a method called _selections(). The inside is going to change, but also the return value, so we're going to have to look at how it's used. Its usage ultimately ends up as an argument to _build(), where it is eventually passed into... _optional_parametric. Huh. Insert le funny Gandalf meme here.
Within _optional_parametric, this value is only used to pass to Part.build() by specifically extracting the values from it...
In there, it's passed to Artifact.convert().

At this point, I kind of lose the plot, because this somehow diverts it through seemingly every part of the artifact module.

...

I think it mainly ends up on the Input type, which passes it back out via exposed_selections(). Which is used for Part.selections(). Which comes back to _selections().

Let's assume we should be replacing MapOverSelection with Parametric[PathStr].

This represents a change in semantics in some fashion, but let's assume the way to update a sequence of these is to filter out incoming duplicates. Maybe look into making callable types for the various Parametric changes, so duplicates can be checked for structurally...

I'll look into trying to do this, maybe in the next few days, maybe next week. We'll see.

Good night.

Coding 2023-04-14

2023-04-14T04:00:00-04:00

Okay, let's take a little time to consider how these decorators are supposed to work. Ignoring MappedParametric for now, because that's a whole thing, I've got two basic options for external interfaces to use:

@_parametric.parametric and @_parametric.adaptive
@_parametric.Parametric.make and @_parametric.AdaptiveParametric.make

Sadly, I do kind of need to consider the lengths involved, since there are valid reasons to use these inline as well as in decorator form.

MappedParametric is kind of an opposite situation, where I'm not quite sure what I want out of the decorator, but there's no reason to care how the decorator works inline when I can just use the default attrs constructor.

I think the first question comes down to how committed I am to having all normal modules import modules instead of importing non-module items from modules. I don't remember the rationale for that, so it's time to try to go back to first principles, apparently. Now, where did I get that idea...

Let's see, my source was certainly the Google Python Style Guide, which, granted, I don't follow perfectly even in this context... Between that, and import this, and a desire to not given in completely to Pokémon-speak, here is my general idea:

Decorate to produce a Parametric with @_parametric.make, as well as a few other module-level functions that can be used as a decorator.
However, also consider adding fluent methods that forward to these functions when applicable. For example, I may want a .flattened method or property corresponding to the flatten function.
Use adaptive for AdaptiveParametric.
Use either one or two decorators for MappedParametric. This is the remaining effort I need to put into thinking about this.

Perhaps @_parametric.map_over(func)... No. Here's a slightly wonky idea...

@overload
def map_over(
    func: Callable[[T], PathStr], /
) -> Callable[[Parametric[T]], MappedParametric[T]]:
    ...

@overload
def map_over(
    parametric: Parametric[T], /
) -> Callable[[Callable[[T], PathStr]], MappedParametric[T]]:
    ...

def map_over(
    func_or_parametric: Callable[[T], PathStr] | Parametric[T], /
) -> (
    Callable[[Parametric[T]], MappedParametric[T]]
    | Callable[[Callable[[T], PathStr]], MappedParametric[T]]
):
    if isinstance(func_or_parametric, Parametric):
        def needs_func(func: Callable[[T], PathStr]) -> MappedParametric[T]:
            return MappedParametric(func_or_parametric, func)
        return needs_func
    def needs_parametric(parametric: Parametric[T]) -> MappedParametric[T]:
        return MappedParametric(parametric, func_or_parametric)
    return needs_parametric

Can't really use it inline, but as I said, you don't need to, and it types cleanly enough, and saves you the bother of having to remember which way around things go by having them both be right.

In fact, I'll try to get it to work now.

...

Hm, it didn't quite work, but it was close. Just needed to make some minor adjustments...

All set for tonight. Later, I can look at this in the light of day, and ponder whether my actions made any sense at all.

Good night.

Coding 2023-04-13

2023-04-13T04:00:00-04:00

I was thinking about how to organize the new code in MOTR, and something occurred to me: That higher-order decorator is really squirrelly, and I'd rather leave it as a module implementation detail than something that gets exposed to other modules. Doing that requires pulling a bunch of code into the parametric module, which is... probably fine? The only real hurdle is that I'd need different names for the "fake value" classes. Again, probably fine. I can stick them inside the associated classes, even, probably.

Let's try that, actually... I mean, it didn't break yet, with the obvious caveat that I'm not really testing these paths.

If it's not broken yet, let's stuff more things in to see if it does...

Well, now all that's left is to figure out the proper way to handle the decorators for MappedParametric. That, and to write the validation for AdaptiveParametric. I should be able to handle that...

And, the validation should be done now. The big thing missing is the last constructor, and instead of rushing into that, I want to take some more time to consider whether to do this as top-level functions, or classmethods, or what.

Like, more cruft at the top level, versus longer access paths on decorators, and is that even the rate dimension to be comparing for tradeoffs...

Not something that I want to rush into.

For now, I'm going to get ready for bed.

Good night.

Diary 2023-04-12

2023-04-12T04:00:00-04:00

I did a bunch of writing and I'm really excited about how it's going, but unfortunately I got so excited that now it's almost midnight, so, um.

Good night.

Weekly Roundup 2023-04-11

2023-04-11T04:00:00-04:00

Wednesday: I started drafting out a new decorator for MOTR.
Thursday: I traced out the usage of some types in MOTR, and realized that I can't take as much out as I want, but I can at least improve the stuff that I have to leave in.
Friday: I tried to extend the implementation of the new decorator to handle the new type I realized I need. I also threw in some Dynamism Crimes.
Saturday: I took care of obligations. Hopefully, I'll be recovered by the time I have to do this again next year.
Sunday: I traveled.
Monday: I came up with some helpful ideas for some writing I'm doing, so that was good.

Next week, I'm going to look for usages of the old Parametric constructors, to see if I can replace any of them with the new decorator. Also, I need to mark up the usages of the old MapOverSelection stuff to make sure I can smoothly replace it.

Diary 2023-04-10

2023-04-10T04:00:00-04:00

Somehow or other, I got some ideas for some of the writing I'm doing. I think they're solid ideas, but I need to review what little I have written currently to make sure I'm not writing myself into a corner.

Basically, I want to make sure that what the main character knows is consistent with what ends up being the case, but to give them as little information as possible within the parameters I'm setting, because there's no reason to make this easy for them.

I'll try to work on that later. For now, I'm just glad that things are shaping up. Now I ought to wrap things up for tonight.

Good night.

Diary 2023-04-09

2023-04-09T04:00:00-04:00

Traveled today, and did a bunch of pointless messing-around.

Maybe before I travel again tomorrow, I'll get some work done on MOTR, maybe I won't.

Either way, I'm going to take things easy, because I think I need to.

Good night.

Diary 2023-04-08

2023-04-08T04:00:00-04:00

I took care of a bunch of stuff today to, um, different levels of quality, but now I should be able take things a little easier.

Right now, though, I'm just kind of fried.

I'm unsure how much I'll get done over the weekend, so my overall plan/goal is to just take things easy for the next few days.

I don't have anything else in mind for this entry, so, it's done.

Good night.

Coding 2023-04-07

2023-04-07T04:00:00-04:00

I thought some about what I want from the other decorator. Something like...

# parametric.py

def make_deco(
    converter: Callable[[Any], Paramatric[object]
) -> Callable[[Callable[..., T]], Parametric[T]]:
    def deco(function: Callable[..., T]) -> Parametric[T]:
        signature = inspect.signature(function)
        instantiators = {}
        metadata = Metadata()
        for param in signature.parameters.values():
            default = converter(param.default)
            metadata = metadata.combine(default.metadata)
            instantiators[param.name] = default.instantiator

        def instantiate(box, selections):
            kwargs = {}
            for name, instantiator in instantiators.values():
                kwargs[name] = yield from instantiator(box, selections)
            return function(**kwargs)

        return Parametric(instantiate, metadata)

    return deco

@make_deco
def parametric(default: Any) -> Parametric[object]:
    if not isinstance(default, FakeValue):
        raise ValueError
    return default.parametric

# another module
def adaptive(function: Callable[..., T]) -> AdaptiveParametric[T]:
    segment_parametrics = []

    @_parametric.make_deco
    def deco(default: Any) -> Parametric[object]:
        if not isinstance(default, SomeOtherFakeValue):
            raise ValueError
        mapped_parametric = default.value
        parametric = mapped_parametric.parametric
        segment_parametrics.append(
            _parametric.map_over_1_parametric(
                mapped_parametric.map, parametric
            )
        )
        return parametric

    parametric = deco(function)

    return AdaptiveParametric(tuple(segment_parametrics), parametric)

It might look a little weird that I'm using the old mapping functions as part of implementing the new mapping functions, but the thing is, there are places where I feel like it doesn't make sense to switch from the old system, so I might as well keep it, and if I'm keeping it, I might as well use it in new places as well.

Aside from the corrections to obvious mistakes I made last-time-ish, I'm now considering what I want the new MappedParametric concept to look like. The "problem" is, both of the things that go into it are functions, unlike the old way, which used a label instead of a parametric. I can't think of which one I'd want to have set up in a decorator position, so maybe... both?

def mapped_parametric(function=None, /, *, map=None, parametric=None):
    none_count = (function, map, parametric).count(None)
    if none_count == 0:
        raise ValueError
    if none_count == 1:
        return MappedParametric(
            map=map or function, parametric=parametric or function
        )
    if function is not None:
        raise ValueError
    return partial(mapped_parametric, map=map, parametric=parametric)

That seems a little cursed, so it would be better to inventory how this would actually be used, before actually putting this in the code and trying to annotate it. (A more restrained way to accomplish something similar would be to create alternative constructors with classmethod, but that's not as funny as putting cartoon swearing in the parameter list.)

Anyway, that's enough for now. I'm going to try to figure out what happened to last night's entry, then wind down.

Good night.

Coding 2023-04-06

2023-04-06T04:00:00-04:00

Before I get into messing with code, let's take a look at what I need to know for one of the future steps, that of getting rid of MapOverSelection. First, let's see how it's used.

In artifact:
- I wrap it in an alias, which is, a bit unfortunate. The alias is used only in the module, which helps. The alias is used:
  - In the signature of Artifact.convert. In this context, it is passed to _selection_labels, which corresponds to one of the metadata fields on a parametric, but I'm not sure which one currently, and to a Condenser and an OutputConverter.
  - In the signature of Condenser.__call__. In this context, it is used directly, to alter a parametric through a folding operation. The probable updated version of this usage is to directly have a set of parametrics, and pass that directly to the fold.
  - In the signature of InternalConverter.__call__. In this context, it is passed to the Input constructor.
  - In the signature of OutputConverter.__call__. In this context, it is passed to InternalConverter.__call__ and the Input constructor, as well as to OutputConverter.__call__.
- Otherwise, it's held in the Input object, and passed out via Artifact.exposed_selections. Which is called from...
The parametric_command module, specifically...
- Put.selections, which is accessed in...
- ParametricCommand._selections, which passes individual items to _update_labeled_maps, and also accesses a similarly-named attribute off of...
- AdaptiveParametric, which dynamically constructs a parametric based off of its attributes, in a very unsafe way, by the standards of this layer. The hope here is to replace this with bare Parametric, or a much lighter wrapper.
- _update_labeled_maps associates a selection label to a MapOverSelection, and this is helpful because...
- It is eventually passed to _optional_parametric? This confusing function only needs to pass it to Put.build, which passes it back to Artifact.convert.

From here, it looks like all I need to do is to make sure some of the types are properly restricted. I think that can be accomplished by...

Using the type Parametric[PathStr]. Or maybe pairs of Parametric[T] and Callable[[T], PathStr]. Which implies to me that maybe I don't rip out quite as much as I was thinking I would. However...

I forgot my train of thought.

Anyway, I think I've got the information I need to plan out new versions. The one wrinkle is, I want to figure out if it's possible to take the core parametric() logic from yesterday, and have some of it in common somehow, since, unless I manage not to need the pair stuff, the pair is going to need its own decorator with an extremely similar implementation.

In any case, I should get ready for bed now.

Good night.

Coding 2023-04-05

2023-04-05T04:00:00-04:00

Okay, the magma is written, and I'll verify that it's "actually correct in general" later, hopefully after I've nailed down some requirements for it.

Let's start working on the next parametric overhaul.

The first half of this has several parts:

A wrapper class around Parametric objects.
A property on Parametric that is documented to return an object of the underlying type, but actually returns an instance of this wrapper class.
A decorator that takes the signature of the function it's passed, and makes sure that all arguments have a default value that's an instance of the wrapper class. Then, map over all of them.

It's probably worth calling the decorator just "parametric". The type should be called, um... FakeValue, or something. Something that hopefully makes it really clear that it shouldn't work for most purposes. And the property, just value, maybe?

I don't know if I want to put these ideas into the actual module yet, but let's see what I can do in code blocks here.

# Eventually, these will be replaced with "attrs.frozen"
@attr.dataclass(frozen=True)
class FakeValue(typing.Generic[T_co]):
    parametric: Parametric[T_co]

...
class Parametric(typing.Generic[T_co]):
    ...
    @property
    def value(self) -> T_co:
        return typing.cast(T_co, FakeValue(self))

def parametric(function):
    signature = inspect.signature(function)
    instantiators = {}
    metadata = Metadata()
    for param in signature.parameters.values():
        default = param.default
        if not isinstance(default, FakeValue):
            raise ValueError
        metadata = metadata.combine(default.metadata)
        instantiators[param.name] = default.instantiator

    def instantiate(box, selectors):
        # I'm pretty sure Python is too strict to allow this, but eh.
        kwargs = {name: (yield from instantiator(box, selectors)) for name, instantiator in instantiators.values()}
        return function(**kwargs)

    return Parametric(instantiate, metadata)

I hope this basically works, disregarding the syntactic gunk that I'll have to work around. I'm going to wrap things up for now, and think about it.

Good night.

Weekly Roundup 2023-04-04

2023-04-04T04:00:00-04:00

Wednesday: I worked on updating the cmd module in MOTR.
Thursday: I realized that the cmd module is going to need to operate in some weird mixed mode for the time being.
Friday: I got the mixed mode stuff working, which gave me the confidence to try to address the Mypy 1 typing issues. Trying to address the issues removed the confidence.
Saturday: I finished with cmd, and had a look at command. That's... even less straightforward, even though there's no legacy path to worry about.
Sunday: I made a small addition to cmd to handle the changes I'm making to command.
Monday: I got almost done updating command. Just a little tweaking left, and it's good to go.

Next week, I'm going to wrap up the command updates, and probably get into some weirder updating projects.

Coding 2023-04-03

2023-04-03T04:00:00-04:00

I was messing with other stuff today, so let's give some quick thoughts to the conversion method I want to update.

The actual update should be pretty simple. I "just" have to update a single type and fix the errors that bubble out...

Okay, that was actually pretty easy, except that I forgot to mark some usages as covariant, so I went add fixed that, and then it all, just, went.

That's addressing the types, but not the new combination logic, which just has to go in command. I don't feel like writing a magma for this right now, even though I basically know what it has to do. There are a few things that I may need to think about. Like, should it be possible to specify duplicate paths in an environment variable, and, if so, do I want some way to opt into deduplication? For now, it's probably safe to ignore questions like that.

Once the magma is in, it'll be trivial to add a new type to parametric_command to handle it.

In any case, the other thing I can be working on is, how to phase out the MapOverSelection stuff.

For now, I'm going to wind down and mess with other stuff.

Good night.

Coding 2023-04-02

2023-04-02T04:00:00-04:00

All right. I've determined that the right way to handle the new environment variable format is to shove everything into the new wrapper types, and use map methods to apply the existing conversion method over the contents.

The wrinkle that I addressed with that was to write new methods to properly handle the whole "sort-of-effects-system" I have going. So, I did that, and now I need to review the conversion method. For reasons, it might make sense to pull it out into its own class.

I'll give this all a shot later.

For now, I'm going to take things easy again.

Good night.

Coding 2023-04-01

2023-04-01T04:00:00-04:00

I went back to the cmd module, and finished updating the interfaces. Everything beyond this point should be a case of switching over completely to the new interface. If I'm remembering and reading the grep output correctly, that should be relatively minor of a change. I think it should just be command and parametric_command.

...

Oof, "just".

parametric_command should be fine, but command is a bit of a mess in this area.

Here's the deal...

command defines an EnvVars type that is parametric over... stuff. It's a map from strings to... stuff. Where "stuff", in this case, is a CmdArg and either nothing else, or an Executable.

(Side note: I find myself wondering whether it's worth having the Path/NonPath distinction, but I still need the new logic to feel comfortable with this code, so let's go ahead with it anyway.)

Now, let's trace what this code is doing. Not going to touch it tonight, but let's trace it.

The type ultimately gets used by resolve_env. This function currently produces a mapping from str to CmdArg. It needs to produce a mapping from str to EnvVar[CmdArg]. Which means that _convert_segment needs to be producing EnvVar[CmdArg], which is a bit interesting, because it doesn't overlap with the other return types from _convert_segment, so I ought to be able to split it out into a new method. At that point, it makes sense to provide three private conversion methods instead of one combined one.

So, the first thing I need to do is to trace out what the other call sites do, and track down which types they need to handle.

That could be a lot. So, I'm going to wrap things up for now, and wind down.

Good night.

Coding 2023-03-31

2023-03-31T04:00:00-04:00

Okay, let's think about MOTR and the Cmd.env field some more. It's arguably inaccurate, but not in a way that matters, to shove everything into one mapping type in each place. So, let's give that a shot.

...

Changing the code seemed to work, but it looks like I made it unhappy...

Okay, that was a legitimate error, a mistype. Fixed easily enough. Now, I'll get the other problem...

And, done.

Now, before I think about continuing the update path here, I should really try to update some of my dependencies.

When I run the latest version of Mypy against the MOTR code base... It complains mightily because of some Extremely Normal code.

...

I slammed into it a few times. Here are the basic issues:

TypeVars that are restricted to a range of values including a (user-defined?) generic type parameterized with Any no longer unify with that type parameterized with a specific type.
Sometimes, for some reason, a function that always defines a return type, is considered to not actually return. The heavy use of overload in this area of code may be confusing things.

I can get around the former case by using an unrestricted type variable, which is basically harmless, but I can't even figure out which line is making it complain in the latter case. Like, yes, there are line numbers in the error messages, but those are at usage, so it's kind of already too late, in that the idea that this function can fail to return is already locked in.

Anyway, I'll get back to that later, or not. I made my choices for the cmd module tonight, and I'll call that good enough.

Good night.

Coding 2023-03-30

2023-03-30T04:00:00-04:00

All right. Starting this entry early, because I've been considering some fiddly changes I want to make within MOTR.

Basically...

The problem with updating the cmd module is that the current release of MOTR is operating at such a low level (relatively speaking, do not scoff) that the motrfiles that I'm currently writing need to reach into MOTR's guts in order to actually accomplish anything. So, even though I have some ideas about changes I want to make (I'll explain those ideas in a moment), I need to make sure that I leave some way for the existing code to manage to accomplish stuff.

So, here's the issue. As the reproduction code I pasted yesterday kind of indicates, I'm doing some stuff where I'm basically replacing T with T | tuple[T, ...]. The problem with this type is that Mypy only allows me to accomplish anything with it, by, um, cheating. The proper replacement is something like NewWrapper[T] | tuple[T, ...] or NewWrapper[T] | OtherNewWrapper[T].

But if I do that, then the core code isn't supposed to handle T any more, which means that existing callers get broken.

So here's the trick: I only need to continue to support the old flow with the cmd module. Everything else is basically invisible to the motrfiles. SO.

I have EnvVar = Unique[T] | Paths[T], StrMap = Mapping[str, T], EnvVars = StrMap[EnvVar[T]], LegacyEnvVars = StrMap[T] | EnvVars[T]. Something like that. What this lets me do is to beef up support for EnvVars until I can cut the internal consumers over to it, but leave the legacy code paths in place until after I finally cut a release.

I probably need to finagle this a bit to account for the fact that I'm specifying concrete types in a few places, but this shouldn't be hard to iron out.

Regardless, I don't feel like I'm going to get it done tonight; I'm just glad I ironed things out a bit in thinking about the way I want the types to be interacting.

That said, I do have the option to make everything a bit less... precise, in exchange for having fewer types kicking around. This will need a bit of sketching to check, and right now I just want to take things easy for the rest of tonight again.

Good night.

Coding 2023-03-29

2023-03-29T04:00:00-04:00

All right...

Let's just do one of these things: the path environment variable work.

Actually, before that, let's see how bad the merge is currently.

...

Not too bad.

Back to the path environment variable stuff.

...

And nearly immediately, I run into trouble.

It was no big deal supporting this at the very bottom level, and I anticipate the levels above will be... mostly workable. What I need to figure out is how to deal with the weird "shapes" that allowing multiple values into the environment mapping is creating. Here's the deal...

The cmd_ wrapper is meant to unwrap values that may be IO-wrapped. (If you know Haskell or a similar language, please try to ignore all possible connotations. They're not the same thing, and if I come up with better terminology, I'll change it.) These values are extract in three different places.

The Cmd constructor needs the same structure, but with all wrapping discarded, and individual values passed through os.fspath. The changes I just made make the structure more complex, and move around the concept of "individual values".
The action (I assume this has a different name in the future topic, but I've forgotten) call needs the stream of individual values, as does the _extra_targets call.

The initial thought that I have is that this implies that the current system of unwrapping doesn't completely make sense. The current system assumes that we can get sensible behavior by passing around streams of values. Now, there's no way around this when it comes to filtering by IO wrapper type, unless whatever alternative system can handle discarding values, but there's no gain in implementing such a system currently.

So, the filtering behavior stands, but the "unwrapping" behavior has to move into a standalone function, and get mapped or comprehended. Let's see...

Write the unwrap function...
Use the unwrap function in cmd_
Remove the method...
Fix a mypy error that I introduced somehow...

Okay, I'm going to need to puzzle over that. But I know approximately what comes next:

Write a function to emit all "individual values" from one of these fancy environment dictionaries.
Write a function to map over one of these fancy environment dictionaries.

I'm going to try to get the error fixed, and then call things for tonight.

...

I sort of understand the error, but it... It shouldn't happen?

I just confirmed that it's not because I'm pinning the mypy version back.

In any case, I'm using a cast to deal with this, but really, mypy should just get the right value itself.

Here is a reproduction. My code is extremely normal.

from typing import TypeVar, Union

T = TypeVar("T")

Thing = Union[str, int]

MaybeTuple = Union[T, tuple[T]]

def untuple(item: MaybeTuple[T]) -> T:
    if isinstance(item, tuple):
        return item[0]
    return item

var: MaybeTuple[Thing]

reveal_type(untuple(var))  # object???

I think I'm going to have to live with this at least until I can actually unpin the version, unless there's some way I'm "supposed to" be setting up these unions.

In any case, I want to wrap up for now.

Good night.

Weekly Roundup 2023-03-28

2023-03-28T04:00:00-04:00

Wednesday: I started drafting the next round of changes to the parametric module.
Thursday: I finished drafting those changes.
Friday: I got some feedback on my writing, and added more esoteric shell gunk to the build script.
Saturday: I dashed out something at approximately five to midnight.
Sunday: I thought a bit about my priorities.
Monday: I made the next round of changes to the parametric module. It went well.

Next week, I'm going to finally get a bit more feedback on my writing, and work on the other overhauls that I feel like MOTR needs.

Coding 2023-03-27

2023-03-27T04:00:00-04:00

Okay, I did the renames to update the fields in the parametric module. The initial update was pretty smooth, although I did run into one weird issue with the attrs plugin for Mypy. I need to take some time to go over the module and make sure the updates all make sense.

One thing I wasn't remembering yesterday is that I want to phase out one of the other modules. Now that I have the new names in, I should be able to coherently reason about the replacement.

I suppose I should also merge up to the future topic, but, eeeh. Maybe later.

So, looking at what's coming next:

Verify name updates in parametric.
Implement path environment variables.
Phase out map_over_selection.
- Implement decorator-based parameter mapping.
  - Write a specialized wrapper class and fake property.
- Document all usages of MapOverSelection; this is going to be a bit of a pain, since it shows up in annotations nearly two dozen times.

Realistically speaking, the path environment variables is going to be less of a pain, but I'm really intrigued by the ergonomics wins from the overall process for phasing out map_over_selection.

Anyway, I can't do any of that tonight, so I'm going to take things easy for the rest of the night, and then work on doing things properly over the rest of the week.

Good night.

Diary 2023-03-26

2023-03-26T04:00:00-04:00

All right, I'm not writing this entry on a stupid time crunch, but I'm going to try to get it done quickly anyway. All day pretty much was taken up with sleep and some important things that we had to take care of.

Tomorrow, I hope to put in work on MOTR, prioritizing things between:

Support for path environment variables
Reworking the parametric metadata label sets
Making any helpful adjustments to how the parametric module is laid out.

The label sets probably come first, and then the other two as I feel like it.

Once all of these are handled, I should take stock of what I'll need in order to advance with work on the wrappers.

But for now, I'm going to put this entry in nice and early, and take things easy for the rest of the night.

Good night.

Diary 2023-03-25

2023-03-25T04:00:00-04:00

Oh wow, I made some bad time management decisions. The best course of action I can take, if I want to publish an entry tonight, is to just publish this tiny little thing.

Good night.

Diary 2023-03-24

2023-03-24T04:00:00-04:00

Okay, so, here's where the writing is...

My wife had a look over the current, very short draft. Things sound mostly good so far, but the experimental stuff is making it a bit hard to follow. So, I'm currently working on adding explanations of it.

The next thing I need to do is to clear up my workflow for generating uploadable drafts, because I'm running into spellcheck-related issues that are definitely an issue on my end rather than any other software.

Basically, because I'm adding a preface, I don't think I want to apply the same spell-checking stuff to the preface as to the main story.

Okay cool I just have to pipe the plaintext version through sed -n '/Preface/,/Main Story/!p' that's not going to be super confusing or anything later I bet.

Anyway, that's stuff accomplished for tonight, cool. It's early, but I'm still tired, so I'm going to wrap up.

Good night.

Coding 2023-03-23

2023-03-23T04:00:00-04:00

All right, let's move through more parametric stuff...

The next function defines a helper, which I'd like to move into the helper module related to "parametric maps". Basically, I want a few different example magma functions, and one of them will be "bail if the values don't match". All together, this should result in something like:

def combine(self, other: Metadata) -> Metadata:
    return Metadata(
        box_labels=self.box_labels | other.box_labels, # I, uh, missed adding this to the other constructors.
        selection_labels=self.selection_labels | other.selection_labels,
        iterated_labels=self.iterated_labels | other.iterated_labels,
        exclusive_labels=self.exclusive_labels | other.exclusive_labels,
        multivalue_labels=self.multivalue_labels & other.multivalue_labels,
        input=self.input.update_with(
            _per_item_parametric_mapping.reject_if_different, other.input
        ),
    )

The next function is... not named well... Let's change it to...

def iterated_labels_except_for(
    self, labels: _selection.Labels
) -> _selection.Labels:
    if not labels <= self.iterated_labels:
        raise ValueError
    return self.iterated_labels.difference(labels)

And now, the last bits of code for the Metadata class:

@property
def singleton_labels(self) -> _selection.Labels:
    return (
        (self.exclusive_labels - self.iterated_labels)
        | (self.selection_labels - self.multivalue_labels)
    )

def selections(
    self, box: _box.Box
) -> typing.Iterator[_selection.Selection]:
    for box_label in self.box_labels:
        if box_label not in box:
            raise ValueError
    # Maybe check that every selection label maps to a non-empty value?
    for singleton_label in self.singleton_labels:
        value = box[singleton_label]
        if len(value) > 1:
            raise ValueError
    return _selection.selections(self.iterated_labels, box)

Let's see, the next really interesting one is "include_box", which. Hm. That should fail if the included box has a non-empty intersection with selection_labels, and it should set box_labels to the difference.

I think that all gets me to a good place with the parametric module. If the rest of the code has a problem with the changes I'm proposing, then that, um... that's a problem with the code that's not in the module, frankly.

Now, I'm going to take a break from working on this for a day or so, then get into either this or the common motrfile concept. In the meantime, I'm going to get in some work on the draft for my writing.

For now, it's too late again, and I need to get to bed.

Good night.

Coding 2023-03-22

2023-03-22T04:00:00-04:00

Well, I'm not sick any more.

So, let's take a look at MOTR's code and imagine updating it.

Going through in order, we have the helper method not_accumulable_because, which is... Only called from not_flex_out_because? Which is only called from artifact._parametric_not_flex_out. Which is used as a validator for artifact.Condenser and artifact.BasicOutputConverter. For now, let's ignore what's going on in artifact, except insofar as this code is actually being used...

Putting all of this together, and keeping in mind that I want to relax the restrictions on multivalue labels, the final result is: one function like so:

def not_flex_out_because(self) -> typing.Iterator[str]:
    if self.input:
        yield "Unexpected use of parametric with associated inputs."
    if self.iterated_labels:
        yield (
            "The base parametric for an Output should not force iteration."
        )

I dropped the restriction on required/exclusive labels, because I think the singleton reforms handle that, as well.

The definition of expected_labels changes to return self.iterated_labels | self.exclusive_labels, and it probably needs a name change.

Next, we have providing and also_providing, which change like so:

@classmethod
def iterating_over(cls, labels: ...) -> Metadata:
    label_set = frozenset(labels)
    return cls(iterated_labels=label_set, selection_labels=label_set)

def also_iterating_over(self, label: ...) -> Metadata:
    return attr.evolve(
        self,
        iterated_labels=self.iterated_labels.union([label]),
        selection_labels=self.selection_labels.union([label]),
    )

Next, we have as_maximal, which maybe needs a name change, but I also have some simplifications planned. Something like...

class Universe(enum.Enum):
    UNIVERSE = enum.auto()

    def __and__(self, other):
        return other

    __rand__ = __and__

    def __rsub__(self, other):
        return frozenset()

    # Don't know if this one is needed.
    def __contains__(self, item):
        return True

Which would be used as the default value of multivalue_labels. So as_maximal becomes:

def as_maximal(self) -> Metadata:
    return attr.evolve(
        self,
        multivalue_labels=self.multivalue_labels & self.selection_labels,
    )

Similar changes to as_requiring:

def as_exclusive(self) -> Metadata:
    return attr.evolve(
        self,
        iterated_labels=frozenset(),
        exclusive_labels=self.expected_labels,
    )

Then we get into narrow_labels, which becomes...

def reduce_iteration_to(self, labels: _selection.Labels) -> Metadata:
    if not labels <= self.iterated_labels:
        raise ValueError
    no_longer_iterated_labels = self.iterated_labels - labels
    return attr.evolve(
        self,
        iterated_labels=labels,
        exclusive_labels=self.exclusive_labels - no_longer_iterated_labels,
    )

And I think I'll leave it there, because something is pretty suspicious about this, in a way that's making me suspect that my model is incomplete or incorrect. There's a few things going on that look suspicious:

Given this logic, which is identical to the previous logic except for the names, I can't tell how the forbid_narrowing concept from elsewhere in the code is supposed to work. Like, this code is, um, pretty insistent...
Oh, wait, I sort of get it. forbid_narrowing can't directly affect what happens to exclusive_labels, but it does reduce the iterated_labels... In other words, "You can't squeeze this if I stomp it flat first." I'm now provisionally okay with all of this except for the names.

In any case, I need to think about this a bit more, then change things if I feel the need, or just move on down through the rest of the methods and functions.

Good night.

Weekly Roundup 2023-03-21

2023-03-21T04:00:00-04:00

Wednesday: I got some work done writing, in spite of some of the software I'm using.
Thursday: I did a bit more work, and reached an important milestone.
Friday: I considered how to move forward with the writing.
Saturday: Hunspell gave me problems.
Sunday: I worked on some of the changes I want to make to MOTR, focusing on planning.
Monday: I got some names improved in MOTR, and locked in the plan to evaluate these changes and then apply them if they work out.

Next week, I'm going to try and make some more progress on these projects. It's hard to say when I'll get started on that or just how much I'll accomplish, because I seem to be at least a little sick right now. I also want to get back to working on the TWaW review backlog. I may need to rethink that project in some way...

Coding 2023-03-20

2023-03-20T04:00:00-04:00

I just did some quick work sketching stuff out for MOTR, and I think I've got what I need to try moving forward with the code:

ghost labels are now exclusive labels
I have a tentative mapping from the old label names to the new label names

I don't think I have much time to work on this tonight, but the next step is to go through all of the operations I have defined on these sets, and make sure that the new names make intuitive sense with what the operations do. In some cases, I'll need to tweak the definitions slightly, to deal with the fact that I'm trying to allow more possible metadata.

The logic for determining singletons is also a bit up in the air. I need to do some combination of set operations on four of these sets.

Anyway, I'll work that out later. Here's hoping it all goes smoothly. I'm going to wrap up early.

Good night.

Coding 2023-03-19

2023-03-19T04:00:00-04:00

Okay, so, I'm writing this late, so this is kind of "Hey, I should work on this more tomorrow," but...

I'm thinking more about the Parametric rework in MOTR, and the more I think about it, the more I realize that I need to nail down some of the terminology.

I'm right now leaning towards four different kinds of data that can go into a command:

Arguments, which have an optional prefix. These have values like --fail-under=100, --show-contexts, -d reports/coverage.
Non-path environment variables. This isn't a great name. It refers to stuff like {"TERM": "xterm"}.
Path environment variables. As above. This is the newest concept in this space, and refers to stuff like {"PYTHONPATH": "/a/b/c:/d/e/f"}, in which the code, not yet written, should synthesize the combined path list from the component paths.
Implicit input and output. Basically, sometimes a command's input or output is not present in the command's text, but MOTR still needs to be able to reason about it. This is stuff like "the command accepts an output directory, but from downstream commands' perspectives, the outcome is that now the file index.html in that directory exists".

These are all collected into a ParametricCommand, and the usual case for ParametricCommand is that every combination of parameter values corresponds to a single command. However, this is not the case for commands that are supposed to aggregate data from several different versions of some other command.

In order to convert the relevant Parametric values in this situation, the values have to undergo a process currently called "reduction". This selectively overrides the default behavior of combining Parametric values. Now, some variables, instead of duplicating and altering the command invocations, are combined within a single invocation. The way combination is accomplished depends on the type of value.

Thinking about this, we can get the following vocabulary concepts:

reduction makes it possible for a Parametric to produce values for some subset of its selections, instead of the full set
combination is the process that allows reduction to occur
some selections must be singleton for various reasons, including in order to implement combination by, um, not combining anything
box labels are all of the labels that must be passed to a Parametric to instantiate its values
selection labels are a subset of box labels, and something will iterate over them at some point
iterated labels are a subset of selection labels: these labels contribute a dimension to the final output matrix

The overall desired behavior for Parametric labels is found by looking at the ranges of behavior for:

arguments, non-path environment variables, path environment variables, implicit IO
static data, inputs, outputs
multiadic maps, reductions

There are two main things here that relatively less obvious:

What controls which labels need to be singleton, and what's the right way to represent this?
What are all of the requirements around implicit IO?

The trick to implicit IO is that it doesn't distinguish the command that is actually run, so any selection label used by implicit IO either must be singleton, or must be an iteration label. Otherwise, the same command will get added to the compendium multiple times with different edge data, which is

bad.

It's like, there's something that will collide with itself unless separated by something else. I don't think ghost labels is a great name for this concept, but it's better than the weird patchwork I have currently.

So, I think the last thing I need to get parity with what the current system models is something that can handle maps over output values. The thing about output values is that any label that any label that an output value doesn't iterate over can't be a non-singleton iterated label.

I'm not happy with iteration/iterated/whatever yet, but I think this all provides a basis to work with. I'm going to wrap up for tonight.

Good night.

Diary 2023-03-18

2023-03-18T04:00:00-04:00

The current situation with Hunspell is Some Serious Bullshit, so if anyone has an alternative to pitch, please reply to this post. Nevertheless, I'm making progress on the draft, so that's good.

I'll be asking friends for feedback over the next few days.

For now, I'm going to take things easy for the rest of tonight.

Good night.

Diary 2023-03-17

2023-03-17T04:00:00-04:00

Filling in details in the writing... I did a little work there, but not much.

I guess I'll have to wait until the weekend to try and make a serious effort. For now, I'd rather try to put all of my conviction that "I should be working on something" into thinking about process improvements or figuring out how to discuss this skeleton of a story with other people.

Okay, I was brushing my teeth and there was one thing I thought of to improve this outline: convert as much of the comments as feasible into [bracketed text] describing what the final version should look like, so the exports have more useful data in them.

I'll start on that tomorrow, because I am tired, and I think I can feel it messing me up right now.

Good night.

Diary 2023-03-16

2023-03-16T04:00:00-04:00

Okay, I've transferred over the important bits from my original "outline", and now I need to work on filling in details or moving forward. And by "now" I mean "later", because I zoned out for quite some time before getting on this, so I need to wrap this up as soon as possible.

Now. Now is good.

Good night.

Diary 2023-03-15

2023-03-15T04:00:00-04:00

The personal writing is going really well, I feel like. I've been refining (and debugging, sigh) the automation for the personal writing I've been mentioning.

"Hunspell is the spell checker of LibreOffice" is definitely a helpful statement that allows you to make sensible predictions about the behavior of the one, given information about the behavior of the other.

Those bumps in the road aside, I spent a bunch of today retyping and updating my initial draft, and, it's like, I guess I'm always going to be happy working on this, because right now nothing else makes me feel like this.

Here's how things look for what to work on now:

Finish retyping.
Fill in whichever bits I want to fill in next.
Work on the motrfile, then work on other coding projects.
Figure out whether I can roll back some of the compromises I made.

Anyway, I should definitely wrap up for tonight.

Good night.

Weekly Roundup 2023-03-14

2023-03-14T04:00:00-04:00

Wednesday: I didn't do much.
Thursday: I started considering some somewhat drastic changes to MOTR. Like, adding new capabilities to core, released types.
Friday: I started to realize just how drastic the changes were.
Saturday: I didn't do much.
Sunday: I started drafting a motrfile that should work with several projects. The hope is to take less effort now than it would take to push MOTR over the line; and less effort than it would take to maintain many copies of a task runner configuration file (regardless of task runner), in order to let me work on other projects, and also to evaluate how I should write the bits of MOTR that aren't written yet.
Monday: I made a little progress there...

Next week, I'm going to work on some writing (which I worked on automation for today), which will hopefully help me deal better with stage fright, which should eventually get us to a point where I can work on the TWaW reviews.

Coding 2023-03-13

2023-03-13T04:00:00-04:00

Okay, let's see. I'm working on a motrfile that should work with the projects I want to work on. It will eventually be available through my Pijul Nest page, but I'm mainly putting it there as a backup, as I don't expect it to be useful to anyone else, unless someone got really excited about MOTR for some reason, and needs a reality check about the current state of the software.

The current state of it is that I'm close to having it ready for running flake8. Next, I'll look into setting stuff up with mypy, then pytest, profiling, and coverage. Next up, shiv or some similar packaging.

The big thing to work out is how to hook up stuff like virtualenv and pyproject-build. This is stuff that I believe should be working in the current development version, but I kind of forget what I did... And, in any case, I can't use the current development version. Or, I don't want to, not until I've come to a decision on those changes I was considering.

Anyway, it's getting late and I'm tired; I should get ready for bed.

Good night.

Coding 2023-03-12

2023-03-12T05:00:00-04:00

I was pretty much just decompressing all day, but I did get some of the work I was thinking of done.

Basically, I want to investigate using the current release of MOTR with the hobby projects that it's supposed to help with. This would be a bad idea if I were writing one motrfile per project.

Which is why my plan is to write the motrfile once in a pijul repository, and then merge that repository into other repositories. The idea is that the functional changes to the code should be entirely orthogonal to the updates I make to the motrfile, so the patches should be completely independent.

Here's how things look for my plans:

Re-implement the MOTR motrfile, but using the lessons I learned writing the higher-level interface, so it's a bit more flexible.
Redo my virtual tabletop project, with a focus first on simpler code, and next on making sure I'm comfortable with what I'm uploading, from a perspective of licensing/fair use. (Nobody reached out to me about this, and I assume nobody knew.) (I may look into aggressively factoring out textual content into external data files that aren't part of the repo. This is more than I think I need to do for some, and seems reasonable for others.)
Make sure the file is reusable by trying it out with stuff like my attempt at Cryptopals.
Find other projects to mess with, now that the main obstacle to using MOTR is hopefully dealt with. (I'm using MOTR instead of Nox because MOTR is, for obvious reasons, better conformed to my mental model for how this stuff "should" work, and also it should be faster.)

The basic goal here is to get around feeling like any Python work I do "has to" be for MOTR. It will make sense to keep working on it, because I want to make it easier to edit the motrfile, but writing it once should be okay, I hope.

Anyway, it's way too late right now.

Good night.

Diary 2023-03-11

2023-03-11T05:00:00-05:00

Okay, I was messing around and let this get too late, so. Not much writing or other stuff today, and I'm thinking I shouldn't try to fit anything in after I publish.

I've been kind of tired lately, so I'm going to sleep in tomorrow, but try to get stuff done once I'm up.

Good night.

Coding 2023-03-10

2023-03-10T05:00:00-05:00

Additional notes on the whole "more ways to handle environment variables" thing.

The correct place to do all of the necessary conversions in probably in cmd.cmd
The place where I need to check for compatibility with that call is in command_builder

I think those are the last missing pieces of the puzzle. Now that I have all of this together, I'm going to take a break and think about whether this is how I want to do things.

I'm pretty sure this all makes sense, but still only pretty sure.

For now, I'm going to wind down.

Good night.

Coding 2023-03-09

2023-03-09T05:00:00-05:00

Okay, I'm not going to commit to making these changes, but let's see what I'm thinking about doing in MOTR. Here's the initial code in the command.py module. EnvVars = pyrsistent.typing.PMap[str, MaybeExecutable[TMaybeExecutable]]. This relies on: TMaybeExecutable = typing.TypeVar("TMaybeExecutable", typing.NoReturn, Executable[typing.Any, typing.Any]) and MaybeExecutable = typing.Union[_cmd.CmdArg, TMaybeExecutable].

What I'm looking into messing with is to allow the map values to potentially contain a sequence of paths, and... hm. Something looks a bit fishy here. Okay, that CmdArg is _io.MaybeIO[PathStr]. So, it should be all right to say "let's add the ability to stick this in a tuple instead, and rewrite the update logic".

And then to actually use this, let's take a look at parametric_command.py It has an EnvVar class, which currently assumes that the values don't really combine. I believe to support this idea, I'd need another class that needs the tuple version of the values, or that provides an adaptor from non-tuple to tuple. Now, that part is seemingly straightforward, but let's see... So, instead of the Artifact in this new class matching the type in question, the Artifact needs to be the basic version, and the build() method needs to... um...

I regret to inform you that... I am confused.

...

Okay, I think it's enough to just change how single items get processed in the other class, and turn them into singleton tuples.

At this point, I'm pretty sure* that if I go through all of this stuff and carry out the "obvious steps", then this stuff "should work".

But, um, "should work" is a scary phrase.

Anyway, I want to wind down now, maybe do some more writing.

Good night.

Diary 2023-03-08

2023-03-08T05:00:00-05:00

Another day where I just kind of took things easy. I'm going to try to write a bit after I publish this, and I have some questionable ideas about MOTR queued up to investigate later.

I wrote this post earlier and then spaced out, so, um, let's wrap it up.

Good night.

Weekly Roundup 2023-03-07

2023-03-07T05:00:00-05:00

Wednesday: I messed around a bit with MOTR's design, and I found... either a new requirement, or a valid relaxation of existing requirements.
Thursday: I tried to approach the requirement changes in a principled way, and, eh. We'll see how this goes when I get back into coding.
Friday: I got a few things done, but nothing that would really fill up a post.
Saturday: I tried to take things easy.
Sunday: I started working on trying out GODEATER, and it went well, until it didn't. Entirely a me problem.
Monday: I tried to deal with that problem, and I guess I'm going to have to keep trying.

Next week, I'm going to either push through the MOTR work, or try out an idea I've been turning around to try and extract some value from it now without making terribly much work for myself later. I also want to take down some of my source code repositories and put up slimmed-down versions, but I don't know if I'll actually get to that. Also also, I'm still doing the experimental writing thing.

Diary 2023-03-06

2023-03-06T05:00:00-05:00

I didn't get too far with the plan to address stage fright, but I did lay the groundwork some, and that went well. I'm using the experimental writing for this, and the experimental stuff still just feels, like, really good. Would probably really annoy most other people, but I deserve this.

Anyway, I should wrap things up for tonight, or I will extremely regret it.

Good night.

Diary 2023-03-05

2023-03-05T05:00:00-05:00

We did some stuff with the core of GODEATER today, and then... I got... stage fright? Anyway, I'm going to need to work through that if we're going to get anywhere with this project.

I have some plans for dealing with that, and I'll try to put them into action tomorrow. For now, I'm going to take things easy.

Good night.

Diary 2023-03-04

2023-03-04T05:00:00-05:00

Welp. I didn't get anything else done today. My plan is to get a bit more done tomorrow.

It's getting late again.

Good night.

Diary 2023-03-03

2023-03-03T05:00:00-05:00

I did a bunch of research and prep work for various projects and ideas, but nothing that translates into much to talk about right now.

I'm going to wrap up now so it's not suddenly midnight.

Good night.

Coding 2023-03-02

2023-03-02T05:00:00-05:00

Taking things slow, thinking about MOTR. I worked out last night that the conditions under which a label is required to be a singleton are somewhat complicated, so there probably needs to be new fields to track the state.

However.

Rather than just throwing fields at it to try to approximate the correct behavior, let's try laying out the final semantics, starting in very general terms.

The most obvious check to make at runtime is, are all of the necessary labels defined?
The next most obvious check is, which of those labels must be singletons?

This second check is trickier. On the one hand, every label not iterated over by every output must be a singleton, so we effectively have an allow list. On the other hand, every label that is iterated over by an environment variable must be either iterated over at the top level, or a singleton. (Inclusive or.) On the gripping hand, every label that is only iterated over by implicit values must be a singleton.

Currently, I believe this last requirement is satisfied by not allowing those labels to exist. There are two "big things" that need to be done in this area. One is to allow all of these scenarios. The other is to give the metadata the ability to notice "hey, here are these selections being used, but forced to be singletons", and emit some kind of diagnostics.

Anyway, I got distracted and it's late, so I'm done for now.

Good night.

Coding 2023-03-01

2023-03-01T05:00:00-05:00

I'm taking a break from GODEATER prep to reconsider what invariants MOTR's Parametric type will want to track. Here's the general outline I need to work through:

The Parametric class exists in order to produce Command objects, which produce a set of Facts relating a Command to other Tasks via Inputs and Outputs.
A Command will have arguments, environment variables, and implicit inputs and outputs (as well as better-behaved values).
There can be any number of arguments.
Each environment variable must have just one value.
Of the Commands produced by a Parametric, no two can produce identical Outputs.
Of the Commands produced by a Parametric, no two can differ only in implicit Outputs or Inputs.

So, some components are outputs, and need to ensure that the actual parameterization does not vary over any multi-valued label that the output does not vary over.

Some components are environment variables, and the final output must vary over at least as many labels as the environment variable.

Some components are implicit, and if they vary, the final output must vary over those labels in a non-implicit component.

I believe these cases cover the existing fields.

These are handled with the methods as_maximal, as_requiring, as well as a few others.

So, we need fields like:

"These fields are the only multi-valued labels that the final output can vary over"
"These fields must be iterated over by the final output [if they have multiple values]" oh no I think I just found a new requirement
"This output is of one type, and these fields need to be iterated over by an output of the other type"

I'm going to need to ponder how to handle the new requirement I just noticed. Or, technically, maybe it's a relaxation of a requirement? A new requirement of what to accept?

Anyway, I'm done for now, but I'm going to note this stuff down so I know I need to tweak stuff.

Anyway, I should wrap up for now.

Good night.

Weekly Roundup 2023-02-28

2023-02-28T05:00:00-05:00

Wednesday: I worked on figuring out what I need to include for a GM emulation reference.
Thursday: I tested out my draft reference, and had some issues with speed. Something to keep in mind for later.
Friday: I figured out most of what I want for reference material, but I had trouble getting it into a small form factor.
Saturday: The printer aggressively did not work, but in a way that isn't immediately obvious, and kept on taunting me with hope.
Sunday: I took a break from this prep work, and did prep work for something else.
Monday: I put together the bulk of the reference material, having solved most of the issues I was grappling with earlier.

Next week, I'm going to take notes on GODEATER, and try to get through a session 0 and maybe a session 1. We'll see.

Diary 2023-02-27

2023-02-27T05:00:00-05:00

I managed to put together much of the reference material I'll need for the first round of reviews, so I'll start taking notes on the actual stuff under review tomorrow.

Beyond that, I don't know, travel and research for other stuff. I took a lot of time to see if I'd think of something else to put, and I didn't, so I'm not going to try to force it any further.

Good night.

Diary 2023-02-26

2023-02-26T05:00:00-05:00

I traveled a bunch today, and didn't start writing this entry until late. I'd been hoping to mess with some code that I haven't touched in a while, but I guess that's not happening today.

I'm messing around with some other writing that might or might not see the light of day. It's kind of a mix of very conventional elements with some experimentation that's very important to me, and the whole point is just to see how I come out the other side.

Anyway, I should get ready for bed now.

Good night.

Diary 2023-02-25

2023-02-25T05:00:00-05:00

Printer... bad...

Nevertheless, I've put together a collection of pages to print out. It's a little... big, but it should get us through next week, when I'll finally start tracking time for this review project.

(Everything before now was what should be one-time prep work, and I didn't want to skew the numbers.)

Anyway, I'm going to try to take things easy for now. We might get a new printer later, but I'm pondering looking for a thermal printer, in the hopes that it won't jam so much. (And also I won't raise my anxiety just looking at the cost of ink refills.)

Good night.

Diary 2023-02-24

2023-02-24T05:00:00-05:00

I got distracted all night trying to come up with a concise format for the reference work I've been doing. I think I've got all of the content drafted, but I'm having trouble condensing it nicely.

Tomorrow, I'm going to print out everything I think I need from the books, and then try harder to condense stuff.

For now, I need to get to bed.

Good night.

Diary 2023-02-23

2023-02-23T05:00:00-05:00

Okay, so, with just a bit more effort, I should be able to put together my concept for reference material. The problem is, I just tried to test run a scenario, and the result was pretty unwieldy. Or, it went fine, although a bit slow, but then I messed with it a bit and lost motivation. Part of the problem was that I managed to get the same plot point twice, which kind of muddled things for me. Hopefully that shouldn't be a common thing using this in practice.

In any case, I've got a draft of the new guidelines for integrating the charts. I'm going to look into two areas later:

Finally setting up the planned flag and highlight layout on the piles of charts I need to print out.
Look into expressing the flows for the guidelines more concisely, and possibly including charts inline somehow.

For now, I need to wind down and take things easy.

Good night.

Diary 2023-02-22

2023-02-22T05:00:00-05:00

Well, I'm not buffering stuff yet. I'm working on the planning for a revision to my reference material for GM emulation. This is a mix of narrowing down Mythic's share to the new core (which is then expanded), redoing my referencing system for the plotlines table, and updating and rewriting the reference material.

While I'm in here, I want to see if it makes sense to write up flowcharts for any of this, and whether it's possible to condense some of these tables onto one page.

I've just about used up my thoughts for the night, so I guess I should get ready for bed.

Good night.

Weekly Roundup 2023-02-21

2023-02-21T05:00:00-05:00

Wednesday: I came up with a writing project: reviewing the contents of the Trans Witches are Witches bundle on Itch.
Thursday: I worked on the planning for it, apparently.
Friday: I began taking notes on the contents of the bundle, to set my expectations for, like, some things are really long, and most things aren't.
Saturday: I finished the first round of notes, and tried to work out just how much effort it's going to take to read all this.
Sunday: I went into a bit more detail in the notes, looking more at the rules.
Monday: I got the notes hopefully complete enough to prioritize things.

Next week, I'm going to focus on redoing some work I did before, to create reference binders for a GM oracle. That way, we should be able to play the games that specify a GM.

Diary 2023-02-20

2023-02-20T05:00:00-05:00

So, here's how it went:

I took a few more notes on all of the tabletop games. Starting tomorrow, we should be good to review the notes and prioritize things. When that's done, then I can work on buffering stuff up.

I can't think of anything else relevant to today, so I'm just going to wrap things up now because it's late.

Good night.

Diary 2023-02-19

2023-02-19T05:00:00-05:00

I'm working on categorizing the roleplaying games still. I'm just kind of classifying some of the rules engagement based on vibes; the main thing missing from this currently is Our Minerva, which I haven't yet had enough focus to properly understand.

The main thing I've determined from the vibes-based classification is that probably none of these have as involved a character creation system as Traveller, which is... probably not surprising?

Tomorrow, I'll try to figure out which multiplayer games to focus on, but I'm going to wrap things up for now.

Good night.

Diary 2023-02-18

2023-02-18T05:00:00-05:00

I've sorted the tabletop games by approximate length; comparison between different games is kind of fake, but order of magnitude is probably generally useful for the ones that are more than a few pages.

After that, I tried reading through one of the games that looked like it was in the middle of the pack. That took about a half hour, so I'll have that as my baseline for now.

One thing that I think helps when I remember to do it is, to set my PDF viewer to 100% instead of fitting width, because having the pages blown up like that makes them intimidating to look at, and hard to read all at once.

The big thing I'm considering now is how we're going to play some of these games. I'm maybe going to have to just review some of them in the abstract, oh well. We'll see if I come up with something else when the time comes.

Right now, I want to wrap things up.

Good night.

Diary 2023-02-17

2023-02-17T05:00:00-05:00

I'm still taking notes on the different entries in the Trans Witches are Witches bundle. I think some of these will end up taking much more time than others, just because of the sometimes-extreme differences in length. Like, if something is over 200 pages long, I expect it'll take a lot more time to review it than something that's 5 pages long.

Also, as I look through these, I have to admit that I'm way more excited about some of these games than others. This isn't, like a commentary on the quality of anything. Like, it's good for a bundle like this to have stuff for people with all kinds of different tastes. It's just, I'll have to see what I think about some of the games that I'm feeling apprehensive or awkward about.

Anyway, yeah, still in planning, so I'm not going to rush ahead yet.

Good night.

Diary 2023-02-16

2023-02-16T05:00:00-05:00

I did a small amount of planning for the review stuff today, and... I have no memory of what else.

That's a shame, but oh well, at least I got something done.

Good night.

Diary 2023-02-15

2023-02-15T05:00:00-05:00

I came up with something to write about. Unfortunately, I ended up spending a lot of time today planning it, so this entry is going to have to be pretty rushed.

Here is the general idea:

Right now, the Internet Discourse around Hogwarts Legacy is Extremely Normal. For my part, rather than spend $60 on a game in a franchise I no longer pay attention to (Johnny Depp was in some of the movies I guess? I don't care?), I put $60 in a related but very different direction. Nobody wants me to provide a take on Hogwarts Legacy, and I don't want to formulate one, so what if I focused on other games? After all, I've got... four-ish tabletop supplements, thirty-eight-ish tabletop games, twenty-three-ish video games, a poem, a soundtrack, and a set of tarot cards in there. I also want to look into a few things not in the bundle that are directly related to it, and a few things not related to the bundle, but I thought of them because of it.

And if this goes well, there are plenty more bundles from the past for me to look through and see what I've got...

So, my plan right now is to pick, say, four things to write drafts on, and see what kind of time frame that gets me. My goal is to figure out some whole number multiple of a week to put stuff out on. If this all ends up being a reasonable set of things to do, then I'll be making posts here, probably under a category of, like, "reviews" or something, and putting reviews on the relevant store pages. I'll be bringing in my wife for additional commentary and for player-two stuff as needed for the tabletop games.

Things are late enough tonight, so I'll wrap up for now, and later I'll finish planning the first round of drafts.

Good night.

Weekly Roundup 2023-02-14

2023-02-14T05:00:00-05:00

Wednesday: I was sick(?) but I was still able to think about MOTR some.
Thursday: Still sick, and I decided to switch gears to interpreter stuff.
Friday: I got a bit further with the interpreter stuff.
Saturday: I relaxed.
Sunday: I came up with some parts of my interpreter code that need to work better.
Monday: Things broke, and it was pretty much my fault, but at least I got some better error reporting out of trying to figure out just what was wrong.

Next week, maybe I'll write tests for the interpreter code, or maybe I'll change gears again. We'll see.

Coding 2023-02-13

2023-02-13T05:00:00-05:00

I'm going to need to thoroughly document how my attempt to put together a higher-level system from a box of scraps actually works, because the failures are all pretty inscrutable.

...

Fortunately, I was able to fix it through the simple expedient of reading the dang code, and realizing that my assumptions weren't lining up with how I'd written one module.

So I fixed that, and things started working. Then I kept on wiring in new modules, and things kept working, and I kind of don't trust it all? I'm going to have to write a bunch of tests to actually verify all of this behavior, and I don't wanna...

Instead, I wrote more code that I'm going to have to test later. Oh well.

All right, it's getting late and I want to wrap up. For now, I'm going to take things easy.

When I come back to this, I should go over all of the modules and make sure I write tests for them.

Good night.

Coding 2023-02-12

2023-02-12T05:00:00-05:00

It can be a bad sign when, while working on code in one language, you try to replicate the behavior of another language. Different languages make different tradeoffs, and this (hopefully) guides the design of what is easy to accomplish in that language. Attempts to import behavior from another context are likely to produce inefficiencies, or even bugs.

So anyway, I've drafted an implementation of Python-tuple-like objects in pure Lua.

...

That's it. That's the joke.

...

Anyway, sure, maybe all of these attempts at immutability are kind of pointless if I'm not going to be changing stuff, but it does also cache the immutable value, which means that identical calls will produce the same table, so hashing and equality will work the way I want them to.

Here's what I want to do next with this code:

Fix up the "frozen class" draft to use this new "frozen array" code.
Figure out how to integrate this with the "enum class" code.
Roll that out where I'm using enum classes currently.
Also, update some of the enum classes to make proper use of the "class" side of things.
I should really bake some reasonable stringification logic into classes somehow. Maybe add a helper function to the disinherit module that can be used to produce a __tostring value.

Anyway, I'll work on that when I feel like it. I'm trying to figure out what I'm really motivated to work on right now, and I'm not sure. I'll think about it some more, I guess.

Good night.

Diary 2023-02-11

2023-02-11T05:00:00-05:00

I'm still recovering from whatever has been messing me up all week, so I'm just not going to bother writing anything substantial right now.

This weekend, I should have some time to mess with coding, because hopefully I'll finally be able to focus.

For now, it's time to relax.

Good night.

Coding 2023-02-10

2023-02-10T05:00:00-05:00

Well, I didn't come up with the greatest interface for the class-enum thing, but it works for now, and maybe I can tweak it later.

Anyway, let's get into what it was for. It looks like I'm getting into things near the beginning of the tutorial I'm looking at. So, let's see...

Earley grammars can be expressed as a list of rules, which have a left-hand side (a single non-terminal symbol), and a right-hand side (a sequence of terminal and non-terminal symbols). Later on, it should also have stuff like a constructor for the non-terminal given on the left, but let's stick to the recognizer for now. (One obvious attempt at optimization would be to store the rules in a table that indexes them by left-hand side.)

Being very careful about reading the example in the tutorial, we see that there's also "groups of (specifically terminal?) symbols".

Anyway, as we move on through the tutorial, we see that there is a concept of "items", which derive from individual rules. It seems to me most natural to build them up by stages, starting with a rule, and then adding the starting index in the parse, and then the progress through the right-hand side. This is because the natural derivation of an item over time is to advance the progress through the right-hand side, but this has no effect on the starting position.

Next, we get to state sets, which I believe I wrote the otable module for. Here it is:

local m = {}

local function non_numeric(k)
  if type(k) == "number" then
    error "Indirect index cannot be numeric"
  end
  return k
end

local function get_by_index(t, i)
  local k = non_numeric(t[i])
  return k, t[k]
end

local function get_by_key(t, k)
  local v = t[non_numeric(k)]
  if v then
    return k, v
  end
end

local function o_it(state, control)
  control = control + 1
  local k, v = get_by_index(state, control)
  if k then
    return control, k, v
  end
end

local function opairs(t)
  return o_it, t, 0
end

m.opairs = opairs

function m.get(t, i_or_k)
  if type(i_or_k) == "number" then
    return get_by_index(t, i_or_k)
  end
  return get_by_key(t, i_or_k)
end

local function add(t, k, v)
  if v == nil then
    return t
  end
  local _, v_existing = get_by_key(t, k)
  if v_existing == nil then
    table.insert(t, k)
    t[k] = v
  elseif v_existing ~= v then
    error "Cannot change associated value"
  end
  return t
end

m.add = add

local function move(src, dest)
  dest = dest or {}
  for _, k, v in opairs(src) do
    add(dest, k, v)
  end
  return dest
end

m.move = move

function m.merge(...)
  local dest = {}
  for _, src in ipairs {...} do
    move(src, dest)
  end
  return dest
end

return require "readonly"(m)

One thing I realize as I look over this code is, I'm going to have to make sure that everything handles __eq, or that I pull some shenanigans to make sure that specific functions produce the exact same values every time.

Another possibility is to just ignore that module for now, and accept some probably-quadratic behavior in the prototype: just implement __eq, and make sure to always scan the target table before inserting.

Okay, I had a proper look over the rest of the tutorial, and realized that I can probably adapt the code from the tutorial into the framework I'm putting together. Here are the big differences:

Instead of text characters for terminal symbols, I'm going to use the existing token class.
I'm going to have to decide how to represent non-terminals, since the actual classes being generated won't correspond neatly to the rules. Probably just using strings would make sense. I want to take some time to consider whether there's anything that would strongly recommend an alternative, so I'm going to take some time to think about this.

Anyway, that's enough for right now.

Good night.

Coding 2023-02-09

2023-02-09T05:00:00-05:00

Overall, I'm doing about the same as yesterday I think. The soreness is kind of moving upwards in my body, and I don't know when it's going to let up.

Anyway, I did manage to remember where I was with regards to the Crafting Interpreters stuff: on a completely different website because I didn't want to do recursive descent stuff. Here's what I ended up deciding:

Instead of trying to have a stripped-down representation for inputs to some of the functions, I had the functions take a more structured format, and make helpers that produce that more structured format. The result is apparently sensible.

I'm going to try to put together the combination of class{} and the new enum{}.

I'll mess with that later, but I don't have the focus to describe anything else.

Good night.

Coding 2023-02-08

2023-02-08T05:00:00-05:00

I'm feeling

interesting

in the aftermath of last weekend, so I can't really focus on any one thing for too long, but I did want to take some notes on another bit of metadata that... I think MOTR currently has, but the names are so bad.

Anyway, there's selections that must be defined in order to produce output, and there's selections that will contribute dimensions to the output matrix. The latter is a subset of the former.

I'm not sure if this information is as important to surface to the user (probably there's some reason to want to calculate how many different commands an input will actually create, although you could also just... run the generation code), but accurately tracking it is vital to making the output coherent.

Still missing good names for these, but my hope is to re-evaluate all of the opportunities for tracking relevant information here, since, as the first two labels I described show, I am currently leaving a lot on the table.

Anyway, I'm going to leave that here for now, and try to relax.

Good night.

Weekly Roundup 2023-02-07

2023-02-07T05:00:00-05:00

Wednesday: I reviewed the existing Parametric metadata in MOTR.
Thursday: I tried to figure out how many things were wrong with the existing metadata.
Friday: I found some metadata that should be in there.
Saturday: I took a look at the helper functions and methods that need to account for the metadata.
Sunday: I had a really bad, like, few hours. I've resolved to do some things in response, that I have not done yet, but I'm working on it.
Monday: I was still kind of out of it, so eh.

Next week, I'm going to try to work on other coding or writing stuff besides MOTR, and tweak plans and specification for it in the background.

Diary 2023-02-06

2023-02-06T05:00:00-05:00

Well, I was traveling today and yesterday, and right now I'm out of it. So, today, I'm going to try to take things easy, and tomorrow, I'll try to act on those ideas from yesterday.

Right now, I just want to relax.

Good night.

Diary 2023-02-05

2023-02-05T05:00:00-05:00

I ended up feeling really bad today, and I couldn't tell what was going on. I was guessing all kinds of environmental factors. Maybe the air was too dry. Maybe it was some dust that I wasn't used to.

But eventually, I think I figured it out. I was pissed off. I was pissed off at the state of pandemic response. That brand new variants are raging all around me, and by and large the measures being taken are woefully inadequate. Basic precautions are advised, then ignored. Massive in-person gatherings.

I am, of course, pissed off at myself for not behaving more sensibly myself, but systemic problems require systemic action. I can keep myself a little safer by having more common* sense in the future, and not behaving recklessly, but the important thing is what happens to everyone.

Maybe I'll get covid, maybe I won't (yet), but the real fucking tragedy of all of this is what the continued pandemic means for people without a support net like I have, and just in general the fucked up way this has gone on for so long, some of the ghoulish things I'm hearing about being done by people with the power to make a difference.

How is anyone supposed to feel when they realize that they, and everyone around them, basically constitute acceptable losses in the minds of the people in charge?

...

The test came back negative, for what it's worth. My run of luck apparently remains unbroken. Good for me. For now.

We shouldn't be forced to rely on luck like this. It's just not right. My heart goes out to the hundreds of millions (hundreds of millions! preposterous!) that we know didn't dodge the bullet, and the families of the millions who didn't make it through.

For the moment, I'll be taking care of myself, because I'm really not prepared to take care of anyone else. And when I'm feeling better, I've got to make some noise.

(PS I rejiggered my upload script a few times to deal with payload sizes and server-side changes so hopefully this actually uploads properly byyyyee.)

Good night.

Coding 2023-02-04

2023-02-04T05:00:00-05:00

Light day today. Here's what I did:

I noted down all of the methods and functions in the parametric module that deal with the invariants I'm trying to work out. One of these, I can probably get rid of, but I'm also trying to work out how to change things up so that other modules stop reaching into the metadata. That should basically be to add more helper functions, although...

Since these functions specifically require their inputs to have the exact right type, maybe I should switch them to methods? Something to think about.

Anyway, it's late and I want to take things easy before the weekend, which I expect to also be light.

Good night.

Coding 2023-02-03

2023-02-03T05:00:00-05:00

I mostly messed with other stuff today, but I'm going to put in a small amount of work on MOTR now:

The parametric metadata should track "the Box must specify a value for these labels". This should be populated by from_box and from_selection, should be left unchanged by flatten, map_over_1_parametric and reduce_parameterization, unioned by map_over_2_parametric (as well as a planned decorator that will replace the map functions in some contexts), and include_box is... complicated. include_box should remove any value that is not iterated over.

So we also need to track "this label gets iterated over", which is populated by from_selection, and passed through everything else in the expected fashion. In include_box, it prevents the removal of the requirement to be explicitly specified.

It's... a little mindblowing that I haven't defined these exact fields before... Anyway, I've written them down in my notes with the date of this post, so I should be able to get myself up to speed when I want to try to implement these.

Anyway, I'm going to call it here and mess with other stuff before I get ready for bed.

Good night.

Coding 2023-02-02

2023-02-02T05:00:00-05:00

I'm not touching MOTR's code, but I did think some about how I want to reason about the different sets of labels, and I came up with some ideas that I think will be helpful.

Considering a Parametric in isolation, there are four basic possibilities when it comes to extracting a value from it:

The metadata is overspecified. This happens when the metadata requires iteration over a selection that is not allowed to have multiple values, but it does have multiple values.
The metadata is underspecified. This happens when the metadata requires iteration over a selection, but nothing iterates over the selection.
The metadata is incompatible with the input Box. This happens when the metadata requires iteration over a selection, but the selection is not defined by the Box.
The metadata is compatible with the input Box. This happens when the metadata is not underspecified, and the Box contains a superset of the required values, which is no overspecified in conjunction with the metadata.

This is helpful to think about, because I'm realizing that some of the information required to track this stuff could be tracked, but isn't.

Also, this is all in terms of selections, but there should be a compatible/incompatible check around non-selection labels. Furthermore, iterating over a label that is added via include_box should render the parametric incoherent.

Fun note to myself because I keep forgetting: include_box prefers the upstream values over the included ones, which means that, actually, it just sort of cancels out and turns into a requirement. Whether this is "the right choice" is a question that, like, needs more usage data to answer. It's hard for me to work out from first principles whether I care about the behavior of "two different values get included under the same label in different parts of the Parametric". One possible mitigation would be to pass the included box up and have a sentinel value for collisions. I'll keep that in mind as something I can do, but the bigger win right now is in formalizing the relationships between the different outcomes. (Like, you can use parametric reduction to stop a Parametric from being overspecified.)

The tl;dr of this is that I need to make sure that I'm actually tracking all of the data required to carry out the proper runtime checks; at this point, I highly doubt that that's the case.

For now, I'm going to mess with other stuff, and get generally towards bed.

Good night.

Coding 2023-02-01

2023-02-01T05:00:00-05:00

Okay, so. I should have expected that I wouldn't be up for much today, but whatever. Let's take a look at the code.

I define four sets of labels that control whether a parametric metadata is valid for various purposes. There is an additional generated set. To extend the scope of validation I need to:

Verify the intended usage of each set.
Update the names to use a clearer metaphor.
Check on the semantics of the helper functions.
Make any required fixes.
Track additional sets.
Create further invariants.

Let's look at the sets:

provided_labels: "the Parametric will iterate over these labels."
required_labels: "the Parametric needs these labels to be iterated over."
handled_labels: "the Parametric can handle these labels being iterated over."
multivalue_labels: "the Parametric cannot handle labels besides these being iterated over." (Technically, it's fine if the other labels don't contain more than one value. I think. At least, it works like that in the code, probably.)
expected_labels: "the minimal set of labels expected in the final product." (the definition is provided_labels | required_labels)

"Product" is actually kind of an interesting word there, because the idea is to form a cartesian product over all of the selections. I don't know if I can work that in. Although, calling them "factors" might help. Or it might not.

Anyway, it's getting late, so I'll have to pick this up later. First, I want to try to explain why there are all of these different sets.

A "provided" label refers to the label's value correlating with an explicit value in the final result.
A label that is only "required" correlates to a value that is implicit in the final result. This is acceptable in intermediate calculations, but leads to problems if it gets all the way out, because that situation corresponds to Cmd objects that only differ in their implicit IO, which is broken sometimes, and I don't think corresponds to any desirable state, so I try to forbid it.
A "handled" label might not be iterated over by the metadata that contains the set, but if other metadata does iterate over that label, then that situation is handled gracefully.
The set of "multivalue" labels, if present, represents a hard limit on the labels that the metadata will allow iteration over. This addresses a similar issue to the "implicit" stuff above, in that multiple Cmd objects should not produce the same output artifact.
A label is "expected" if either it is explicitly iterated over, or if something else should iterate over it, even though this metadata doesn't.

That's a lot to unpack; I'm looking at this and kind of wondering if sorting this stuff out would be a good use case for formal methods. For now, I've written these field names down and suggested clunky but descriptive new names; the goal from that is to come up with something better than either, in each case. And now that I've done that, I'm going to get ready for bed.

Good night.

Weekly Roundup 2023-01-31

2023-01-31T05:00:00-05:00

Wednesday: I started changing my mind about how to reorganize artifact.Output.
Thursday: I noticed even more ways I could potentially simplify the code...
Friday: I took a little break to ponder effect systems.
Saturday: I started planning ahead to the next set of changes I want to make, which I want to make because they mean deleting code, which means I wouldn't have to document it.
Sunday: The MOTR update got... a little out of hand.
Monday: I got the MOTR update under control, and then started planning for other things.

Next week, I'm going to look over the parametric metadata fields and try to improve them.

Coding 2023-01-30

2023-01-30T05:00:00-05:00

I got the Output stuff updated and qualified. The merge was shockingly uneventful, all things considered.

I could try to work on several different fronts now:

Try to fix the new names I came up with, because there's a bunch of "well I needed to call it something" classes, and they really don't make things clear.
Try to add new wrappers using the current state of things.
Remove the MapOverSelection class.

I really want to do that last one, then loop around to the beginning. It's late now, so I'm going to just quickly lay out some knowledge I'll need later:

parametric.Metadata currently does not track box.Label status; I did the hard stuff first with selection.Label.
box.Label should be just "here are the labels it needs".
The selection.Label stuff is going to need the names re-evaluated. Like, "provided", "required", "handled", and "multivalue"? I could try to defend those names, but I guarantee it wouldn't be persuasive.
Once I re-familiarize myself with the selection.Labels meanings, I should be able to figure out the required relationships between the metadatas in what is currently called AdaptiveParametric.
Also, the different parts of parametric_command should be named to properly correspond to the unique/parametric/artifact distinction.

Okay, I did a little thinking, and I really need to figure out how selection.Label stuff should interact with include_box, and I'm still on "yeah, it probably shouldn't allow that". But I should figure out how to express that in terms of the existing metadata.

One last note: once I have this all converted to use Parametrics, then I should be ready to assess whether to try doing the ContextVar stuff.

For now, though, I should get ready for bed.

Good night.

Coding 2023-01-29

2023-01-29T05:00:00-05:00

Uuuuuuugh.

I ran into, and continue to run into, issues with updating the artifact module.

I have been bouncing off this all day, and I've made a lot of progress, but getting past this last bit is a pain because it's confusing. This is exactly why I wanted to do this kind of work before I consider adding more wrappers.

There's a lot that happened, but the short version is that my initial design got sort of fractured into even more classes, and some of my assumptions got challenged, but that part is okay. What remains to update, as far as getting the code to typecheck, is a worryingly large function that is just two statements.

...

I've got something that looks like it fixes that function, but now I've got other weird stuff to fix.

...

And, it's fixed, thanks to... stuff.

Sometimes I can understand typing stuff, but other times, eh.

Here is the plan for later, definitely not now:

Get flake8 to pass over the code, because I was not bothering to fix style issues.
Get pytest to pass over the code, because it seems extremely likely that I regressed something.
Confirm that the old code paths are dead and I can delete them.
Mer...

Mer...

merge

Y-yeah...

I can tell from here that that's going to hurt.

But for now, I'm just going to wind down and get to bed, because I need rest after all of that.

Good night.

PS I had to walk back the glitchiness on the smiley because the glitchy versions don't look as good on my new laptop, and I'm not sure what to do about that. I guess there's a theme update coming down the pipe.

Coding 2023-01-28

2023-01-28T05:00:00-05:00

All right, I'm not touching the code for MOTR, but I want to plan things out for later.

Here are the modules that should guide my changes later:

artifact
parametric
parametric_command

The goal is to eliminate the map_over_selection module. I've got a two-ish part plan to do this. First, as I've said, I want to rework the artifact module to stop using the *Accumulator classes, by reorganizing the Output class. The other part is to replace the MapOverSelection class. The MapOverSelection basically associates a Label with a function that maps that label's associated value to a PathStr.

The thing I realized is that this is basically equivalent to a Parametric[PathStr] with a bunch of restrictions on its metadata. My goal is to eliminate the usages of this class by figuring out the actual metadata restrictions, and having the code require those restrictions, where applicable. This will actually make the code more reliable than it is now, because some of those restrictions are currently enforced, at best, by convention, which makes this code even more useless than one might reasonably expect for anyone who isn't me.

Here's some background:

There's sort of a latent mental model I have, of a division of the files that MOTR works with, into two categories: source, and artifacts. Source files are the background or basis for the operations that MOTR carries out, and are expected to be tracked in a source control system like Mercurial, Pijul, Darcs, Fossil, Bazaar, Perforce, Subversion, etc. Artifacts are anything that MOTR creates based on its configuration and the source files. This includes generated constraint files, static analysis, test results, coverage reports, profile data, and, like, anything else that could make sense there.

The parametric_command module divides the paths that are fed into command constructors into three categories:

artifacts
unique source files
groups of related source files

The first two groups are in better shape than the third. The way the third group is represented is a bit wonky, so I'm just going to describe what I think it should be doing:

There is a Parametric[T_co] and one or more Parametric[PathStr]s. All Parametrics that are any more than mildly complicated have metadata relating to selection labels. These two groups of Parametrics must have metadata requirements that are, um, complementary? Basically, the latter group feeds into the pipeline that creates artifacts, so whenever there's a different value from the Parametric[T_co], the Parametric[PathStr]s should collectively have a different value.

This system kind of breaks down if the functions that make up a Parametric end up mapping two possible Selection values to the same output, and my advice is to simply, not do that. This, kind of sucks, but I have no idea how to express "this function must be injective" as a Mypy-compatible annotation. To be fair, I don't think this is a particularly unique shortcoming of Python.

Anyway, it's getting late, and I'm getting tired.

Good night.

Coding 2023-01-27

2023-01-27T05:00:00-05:00

In the previous entry, I got diverted into thinking about whether I want to try using ContextVars in some places that I'm currently basically saying "pass through these parameters everywhere". The reason I'm pondering this is because I've read Koka's documentation, and I saw some connections between concepts in Python that don't obviously map to "the same underlying thing". This post could be pretty rough.

Here's the background: Koka uses effects for a lot of things, and provides various possible behaviors that an effect handler can apply to the effectful code. Basically, part of an effect in Koka is a set of functions that code that has the effect can call. The result of calling these functions is determined by the dynamic scope of how the function is called: most effects are not handled by main(), so they have to be handled by other code. A handler is an expression (I think...) that evaluates to a value of some type. This is done either by having its associated block execute to completion, or as part of an implementation of a function required for the effect.

The simplest thing to put in a handler implementation is an expression that does not resume the effectful computation. Let's compare the example from the Koka documentation with the equivalent code in Python:

fun raise-const() : int
  with handler
    ctl raise(msg) 42
  8 + safe-divide(1,0)

If we ignore the fact that the Koka documentation is defining some effects and functions explicitly for pedagogical purposes, the equivalent in Python is

def raise_const() -> int:
    try:
        return 8 + 1 / 0
    except ZeroDivisionError:
        return 42

There are some syntactic things that stick out, like that Python doesn't have an equivalent to Koka's with, so the indentation is, like, more severe? (Even ignoring that I 2-spaced Koka. Count the levels of indentation between the def and the +.)

The next thing the Koka documentation talks about is resuming, and I'm actually not sure what the idiomatic translation of their basic examples is. The reason I'm not sure actually gets into why I'm interested in working with effect systems.

Here's the Koka code I'm looking at...

effect ask<a>
  ctl ask() : a

fun add-twice() : ask<int> int
  ask() + ask()

If I wanted to surface the resumptive behavior to Python's type system, I think I'd end up with something like...

T = TypeVar("T")
R = TypeVar("R")

Ask = Generator[None, T, R]

def add_twice() -> Ask[int, int]:
    return (yield) + (yield)

But I could also hide this distinction from the runtime (and, as a consequence in this case, the type system) using ContextVars:

ask: ContextVar[Callable[[], int]] = ContextVar("ask")  # need to hardcode result type

def add_twice() -> int:
    return ask.get()() + ask.get()()

The latter option definitely looks a bit strange to me, but the reason I'm interested in it is because of function color. I can either give an example that works with these functions, or make them look a little more realistic...

def add_to_input_gen(var: int) -> Ask[int, int]:
    return var + yield

def add_to_input_ctx(var: int) -> int:
    return var + ask.get()()

(I did say a little.)

The point is, these new functions, at least conceptually, can be passed to higher-order functions or nice syntactic constructs. Let's pretend we have a reason to want to add numbers to a stream of data, but we want to be able to dynamically change the logic that generates the added numbers. For the latter function we do...

def add_to_stream_ctx(vars: Iterable[int]) -> Iterable[int]:
    return map(add_to_input_ctx, vars)

For the former, we, um... Look, I know I sometimes (very rarely) pretend to not notice something for attempted pedagogical purposes, but I'm really not sure what you'd do. I want to be missing something, but when I try to sketch something out, it just turns into a mess.

There's a lot more to Koka, but it's a combination of stuff that relatively-straightforwardly follows from all of this, or stuff that I can't imagine working well in Python, like the ability to resume from the same point multiple times.

For completeness, I'll try to translate my "more realistic" code back to Koka:

fun add-to-input(input : int) : ask<int> int
  input + ask()

This signature is in some sense even more specialized than the Ask[int, int] stuff, which was, in the end, nothing but standard library types mushed together. The difference is the higher-order functions in Koka's standard libraries are generic over effects. This gives you guarantees like "mapping over a total function is total" and its converse, "mapping over a non-total function is non-total". Or use custom effects like ask<a> explicitly. To do that in Python, you either need a new higher-order library, or you need to make Python "not see them", except to the extent that you need to provide implementations.

(Like, I didn't actually set the ContextVar anywhere, and if you don't set it, then, um, it's going to fall over in a way that I believe is imperceptible to static analysis.)

I'm interested in all of this not because I really want to choose different numbers to add to other numbers under various circumstances (at least, that's not the main thing I want), but because it seems really neat to be able to say "Here is a shiny new concurrency library. It is fully compatible with the higher-order functions in the standard library."

For this post specifically, I just wanted to think about some of the ways that Python surfaces the concept of effects, and how they don't really gel in Python, and I'm not really sure they could be made to gel in Python without a compatibility break that would make the transition from 2 to 3 look like a tea party.

And that's why I want to learn OCaml. (I haven't tried very hard yet, but I will.)

Good night.

Coding 2023-01-26

2023-01-26T05:00:00-05:00

All right, let's see where I want to take MOTR's artifact module. The key thing I realized thinking about the reorganization is that I can corral some functionality into the "constructor" idea I had, so it can be a helper class that, instead of unpacking all of that into an Output, it gets included into the class, and delegated to for the following flow:

Given a sequence of MapOverSelection[object] objects...
Produce a Parametric[io.Input[T_co]] object.

So, what do I want to call this?

(Horrible diversion: there are legitimate reasons to try to retire the concept of the MapOverSelection and make this code work directly with Parametric[PathStr] objects, but oh boy I do not have the bandwidth for that currently.)

Actually, can I run with that parenthetical? It takes several Parametrics and, I want to say, it condenses them into one?

"Condense" is the word that my gut wants me to use, and I need to ponder what sense it has in this context. I think the reference is to the idea of condensing a piece of writing, and not directly to anything involving physical substances.

I guess what I'm wondering is, can I come up with a nicer name than Parametric? Aside from the classes that I'm actively attempting to remove in the current effort, Parametric seems like pretty low-hanging fruit. Its point now is that you give it a Box and it uses metadata to create Selections from that Box, and passes them in together to a function.

Slightly alarming idea: I've been looking at some of the code I've written, and wondering if using ContextVars would be a cleaner way to handle shared data/behavior.

A somewhat-related idea that occurs to me is that the implementations for, um, Parametric.instantiate, in some sense, "should" be written with the keys they access in their signatures somehow. Maybe something really straightforward like def instantiate(box_var: Annotated[int, SOME_BOX_INT_LABEL], selection_var: Annotated[str, SOME_SELECTION_STR_LABEL]) -> ResultType:.

I don't have a sense for whether these are unambiguously good ideas I'm tossing around; they do seem like they would improve some things, but I can't tell if there are any major caveats.

...

I took some time to trace the implications of what I want to change, and they're frankly too complicated. If I can convince myself that some invariants that the code currently maintains don't actually matter, then I can cut down on the number of classes used.

I think the play to make is, this weekend, put together the required changes to artifact, and try not to worry too hard about the names. Then, switch to the documentation branch and start documenting the invariants of the parametric, parametric_command, and artifact modules.

My suspicion is that those modules in particular are dragging around some outdated simplifying assumptions. If I can nail down what those modules need, I can work out the feasibility of cutting down on the number of high-level concepts in the code.

Independently of that, I also want to mess around with context variables a bit, because I think they can substitute for most but not all of the uses of a heavily-forked variant of the punq library I'm using in another hobby project.

I think that's all for now.

Good night.

Coding 2023-01-25

2023-01-25T05:00:00-05:00

Okay, taking another look at how I want to tweak the construction of artifact.Output. Basically... There are three things exclusive to artifact.Output:

A seed Parametric that currently has to produce a PathStr.
A Callable[[PathStr, PathStr], PathStr] to combine other values into it.
A Callable[[PathStr], T_co].

Currently, the first two get bundled up. I'm thinking I might want to bundle the third one in with the others. It's either bundle stuff up more, or less, and I don't know which way is better to go...

I'm going to try to bundle them up, but today, and the next few days, I'm a little booked as far as things to do. I hope to try this out sooner, but it might end up waiting until the weekend. We'll see.

I can't delay things any longer.

Good night.

Weekly Roundup 2023-01-24

2023-01-24T05:00:00-05:00

Wednesday: I kept on renaming stuff in MOTR.
Thursday: I finished the name updates, and then dealt with the merge.
Friday: I planned some test speedups and coverage improvements.
Saturday: The speedups worked.
Sunday: I sped things up a little more, and dealt with the merges.
Monday: I did some evaluation and planning of how I want to improve artifact.Output, and I've got some interesting ideas, for sure...

Next week, I'm going to try to execute on my latest plans for improving MOTR. After that, I should be good to try implementing more wrappers.

Coding 2023-01-23

2023-01-23T05:00:00-05:00

All right, I want to figure out how to redo the artifact.Output class so it's like, less janky, but it's not obvious to me how to get from here to there. Let's take a look at what it has and what I want.

@attr.dataclass(frozen=True)
class Output(Generic[T_co]):

    input_accumulator: "This is basically an initial Parametric[io.Input[PathStr]] plus a fold operation over PathStr. Going forward, I want to break this into two fields."
    map: "A function from PathStr to T_co. I'm going to keep this, and it should be helpful in eliminating some of the weird cruft I ended up adding."
    output: "This is misnamed; it is a Parametric[Sequence[str]]. It should probably have a default value that just evaluates to the empty tuple. Its name should include 'name' somehow."
    target_label: "I don't know if 'target' is the right word, but the only change to make here is to probably make it optional."
    make_parent: "Here's the big change. This is currently an optional data-y value that produces various kinds of outputs. I want to replace it with a data-y type that combines a function from T_co to a Datum subclass with an InputLabel of that subclass, and has a helper function to wire that all up in the only way that really makes sense."

...

# So, the dream is something like...

TDatum = TypeVar("TDatum", bound=Datum)

# I have no complaints about the new attrs module, but I'd rather wait to
# switch until after I've stopped maintaining two topics in parallel.
# Also, maybe I need to make TDatum covariant or something.
@attr.dataclass(frozen=True)
class ExtraImplicit(Generic[T, TDatum]):  # Formerly, this would have been "ExtraExtra" 🤢
    label: InputLabel[TDatum]
    map: Callable[[T], TDatum]

# Fairly straightforward helper functions go here-ish.

@attr.dataclass(frozen=True)
class Output(Generic[T_co]):

    initial_parametric: Parametric[io.Input[PathStr]]
    pathstr_accumulator: Callable[[PathStr, PathStr], PathStr]
    map: Callable[[PathStr], T_co]
    names: Parametric[Sequence[str]]
    input_label: InputLabel[T_co] | None
    extra_implicit: tuple[ExtraImplicit[T_co, Any], ...]

My main concern, looking at all of that stuff, is how to put together a sensible constructor interface. Part of me wants to just slap a basic fluent interface over top of it all and call it a day, but that doesn't feel good to the rest of me. Perhaps if I consider what the other majorly reworked class would look like...

@attr.dataclass(frozen=True)
class OutputFromInput(Generic[T_co]):

    input: Input[T_co]
    names: Parametric[Sequence[str]]
    input_label: InputLabel[T_co] | None
    extra_implicit: tuple[ExtraImplicit[T_co, Any], ...]

This implies that, from a code organization perspective, we need a class representing "how a Parametric[io.Input[T_co]] gets converted to a Parametric[io.Output[T_co]], and, with the help of a callback, then to a Parametric[command.Metadata[installer.TArgs]], although, I've just realized...

I need an additional option on this thing for handling the "make parent" case, because that's an implicit input rather than an implicit output. So, the helper class is something like...

@attr.dataclass(frozen=True)
class OutputConverter(Generic[T]):

    names: Parametric[Sequence[str]]
    input_label: InputLabel[T] | None
    make_parent: ParentMaker[T] | None
    extra_implicit: tuple[ExtraImplicit[T, Any], ...]

ParentMaker = Callable[[T], pathlib.Path]

Or something. I don't know if there's an idiomatic way to have "a boolean variable that can only take a True value if a type parameter has a specific value"... Like, this is seeming like a really tempting case for subclassing, even by my standards. I suppose the alternative would be to define a protocol based on its inputs and outputs, and write a wrapper class around the basic converter, then add a fluent method that requires T to be pathlib.Path, and then... yeah, at that point, it should Just Work.

I'm not getting this done tonight, but I'm glad I've got a pretty solid plan now.

Good night.

Coding 2023-01-22

2023-01-22T05:00:00-05:00

I squeezed out the last two seconds of speedup left on the table, and grabbed about four more. After that, there's about 4 that I can get later, but I can't get now. The merges required are getting a little frustrating, but it's probably fine. I also put up the coverage a little. Currently hovering at or above 97.5%.

I'm guessing that both of the things I want to work on next will do something horrifying in terms of merge conflicts, so I'll just try to focus on what should be the smaller one.

That's going to be basically taking apart the artifact classes and putting them back together in a way that makes more sense.

For now, I'm going to wrap things up.

Good night.

Coding 2023-01-21

2023-01-21T05:00:00-05:00

I ended up "only" speeding things up by ten or so seconds, but that's not nothing. I think there's about four seconds I can save easily, and four more that I can take care of down the line.

As far as what to do next, wiping out the coverage misses that just got opened up is a solid choice, but I also want to look into the artifact.Output rework.

Anyway, for now, I want to zone out for a while, then go to bed.

Good night.

Coding 2023-01-20

2023-01-20T05:00:00-05:00

I forget how or whether I prioritized things for MOTR recently. Now that I've got a round of renames done, I was like "what do I feel like working on?" and the answer was apparently "remove the usages of pyrsistent that aren't PMaps". Which, I wanted to do that at some point. It might have been prudent to do it on the "future" side, since the merges were all gross, but, eh... Offhand, I can remember a few other things to look into:

Reduce the dependence of tests on subprocesses.
Rework the artifact.Output layout and creation workflow.
Work on splitting up the API into logical layers.

Looking over these, the first is good to do when I feel like it, since it shouldn't involve a bunch of soul-searching or wide-ranging changes, and getting it done should shave a few seconds off of the run time of the tests, which is several percent improvement.

The other options both take some careful planning, and I wouldn't be surprised if they caused interesting things to happen in terms of merges.

...

Wait a sec, I was going by memory for the expected speedup, and it's more like, if I can get just one test to run at a similar speed to nearly every other test, the tests will be nearly three times faster?!?! However, it's right in the middle of "future" changes. At the same time, getting the logic relevant to this test mocked would make it easier to increase coverage, so I think implementing the mocking twice, more or less, would be less painful than trying to legitimately force a failure state to trigger the unused branches.

All right, this is clearly the play. It speeds up iteration, and gets me closer to full coverage. Then I can work on these other changes. Probably planning out the different layers as I feel like it, but focusing on the artifact rework.

Anyway, I should get ready for bed.

Good night.

Coding 2023-01-19

2023-01-19T05:00:00-05:00

I had a rough day at work today, but now I'm

FINE

Because I got the name updates finished in MOTR, and then I merged them, and then I fixed the merge conflicts, and then I updated all of the things that the merge didn't update, and when I'm feeling a little less tired and twitchy I can start working on more significant updates.

Right now, though, I desperately need to wind down.

Good night.

Coding 2023-01-18

2023-01-18T05:00:00-05:00

I have been so in the zone propagating renames through MOTR's code base. I'm stopping and writing this post, not because I got to any kind of natural stopping point (I absolutely did not), but because I don't have it in me to update any more code tonight.

I ended up stumbling across some really nice improvements to make while I was working through these files, and I have a few ideas for further tweaks to try out later.

For now, I need to take a few more days to get these renames done, then to run flake8 on the result, then to test the results. Then merge. And probably do this all a few more times.

Progress... Yay...

Anyway, it's late.

Good night.

Weekly Roundup 2023-01-17

2023-01-17T05:00:00-05:00

Wednesday: I was focused on the CtC hunt.
Thursday: I finished the hunt.
Friday: I started thinking about MOTR again, and getting bits of the code to make more sense.
Saturday: I didn't get much done.
Sunday: My face hurt.
Monday: I started making changes to MOTR.

Next week, I'm going to try to get MOTR to a state where it can pass its tests again, then merge to the future (which will suck), then make a few more changes, then see if I feel ready to write more support code.

Coding 2023-01-16

2023-01-16T05:00:00-05:00

All right, I didn't really focus on much today, but I did work out something to get more work done later. I did the beginning of a bunch of naming updates.

So, now the new code is pretty thoroughly broken (though I plan to break it more), and I can easily call up a list of everything that needs to be updated.

So, with these changes, it's simple: fix type issues until it thinks things work again.

I can't think of anything else, so I'm going to make a few more name changes, to queue up the checklist for later.

For now, I need to get to bed.

Good night.

Diary 2023-01-15

2023-01-15T05:00:00-05:00

Oh man, I feel so bad right now. Like, my sinuses are doing something that they definitely shouldn't.

I don't feel up for anything too intense, but I did take a few notes for the Other Project. And I can probably handle taking paper notes on MOTR changes, now that I've managed to grasp how the scope should be narrowed.

But other than that, I need to make a sincere effort to do better at winding down.

Good night.

Diary 2023-01-14

2023-01-14T05:00:00-05:00

Took another light day.

Over the weekend, I'll hopefully have a chance to lay out all of my thoughts for MOTR and get them to make sense, but right now, I should wind down.

I did at least put together some stuff for the Other Project. At some point, I should figure out if the Other Project needs, like, legal advice. But that's not right now.

Good night.

Coding 2023-01-13

2023-01-13T05:00:00-05:00

Instead of touching any code today, I'm going to try to fully lay out my thoughts on the general area I was talking about last time.

For context, I'm trying to rework the structure of the class that will be call artifact.Output once I've renamed everything. The various classes in the artifact module are (okay, fine, will be) implementations of the Artifact protocol, which defines a method that takes a callback which converts some value to a Parametric[command.Metadata] (or something along those lines. artifact.Output currently has provisions to inject other values in specific circumstances, which end up making it hard for me to determine which intermediate values are "supposed to" be used at various stages.

Now, I was thinking of putting together something really elaborate to replace this system, but I think the specific really elaborate thing I had in mind had its own problems. So, let's take it slow...

The current system is required to work with pathlike objects for the "accumulation" phase that happens before anything else. It also has an optional "mapping" step that's required to stay pathlike, and could in theory be removed, so eh.
Once the final pathlike is produced from the accumulation, an additional mapping step is carried out, and the relevant "PCM" object is constructed using the callback.
The instance may contain information relevant to adding information about the parent path of the "final" pathlike to the "PCM", and if so, that pathlike is passed in. The mapped value cannot be used, because its type is not statically known.
This could all be drastically simplified and address some of the problems, except for the case currently referred to as Extra (which will be Implicit). The callback passed in that case accepts a type that partially overlaps with MOTR's definition of pathlike, and we really want to be able to use the types outside of that overlap. While the type in question sometimes does contain path information, at least sort of, it's not guaranteed.

So, going from there... I don't think there's a sensible definition of "parent" for these "Token" objects, but unless I can formalize the types in question, I need the code to do something when this overlap happens, and I'd rather not stick isinstance checks in there.

So, I want to change this code both from a usage perspective, and from an implementation perspective. From a usage perspective, I want the "final" accumulated value to be what is passed to the command if possible (so, ideally not Extra/Implicit), and I want the other values to be Extra/Implicit.

This actually really helps, because it gets me that these extra values are only expected when the produced value is known to be pathlike. So, if there are update methods that restrict their validity based on the type parameter, then those methods are allowed to use the fact that the data is pathlike. I don't know if I should exercise that option, but it's there.

Anyway, a few other notes:

Only the "explicit" value needs to be named, because the names will always sort of refer to the same things.
However, it would be good to allow any output to have its own label.

These involve maps like PathLike -> T_co and T_co -> Datum, but the latter is only expected with T_co == PathLike.

Something else I need to keep in mind is that I believe I want to add these capabilities to the OutputFromInput class, which... hm... Basically, all of this logic "should" be neatly encapsulated somewhere, and the question of how to do that feels like maybe it should wait for after I've done the previous rounds of rewrites.

So, the plan for the coming days: take my existing plans, convert them into TODO comments in the code, then grind through them and take notes on what to do differently.

For now, though, I really need to sleep.

Good night.

Diary 2023-01-12

2023-01-12T05:00:00-05:00

Hunt is done. I kind of want to jump right back into other stuff right now, but that would be a bad idea.

Aside from the hunt, I did take some notes on the other project, and now I'm ready to start planning how I'm going to actually produce stuff for it. I'm thinking I'll try to figure out what kind of techniques I want to work with, and use them with other topics. We'll see.

Anyway, I should wrap up for tonight.

Good night.

Diary 2023-01-11

2023-01-11T05:00:00-05:00

All right. I was still focused on the CtC hunt, so no hobby coding stuff. I'm closing in on the end of it, but there's still one and a bit puzzles to go.

I'm also gradually assembling resources for The Other Project, which I will properly announce at some point. I just don't feel comfortable saying "Here is my idea for a thing I'm doing. If that sounds interesting... too bad, lol, check back in like a year." So just, trust me that I'm trying to put something out, and I won't make people wait for anything specific until after I've put part of it out.

In any case, it's too late currently, and I'm not going to get anything else written tonight, so this is done.

Good night.

Weekly Roundup 2023-01-10

2023-01-10T05:00:00-05:00

Wednesday: I came up with a possible reason that I'm having trouble with one of the classes in MOTR.
Thursday: I mostly focused on the Cracking the Cryptic puzzle hunt.
Friday: I looked a bit more at MOTR, and tried to figure out how to make some of the code make fewer assumptions, in order to be clearer.
Saturday: I tried to work on another project...
Sunday: I dunno.
Monday: I roughed out some possible ways to rework the artifact implementation.

Next week, I'm going to try to flesh out the other project, and take notes on changes to MOTR.

Coding 2023-01-09

2023-01-09T05:00:00-05:00

All right. I got some work done on one project today, let's sketch stuff out for MOTR.

My idea for generalizing artifact.Output is to have a helper class that looks something like

@attrs.frozen
class Converter(Generic[T]):
    label: Label[T] | None
    names: Parametric[Sequence[str]] = attrs.field(
        default=no_parameters(()),
        validator=RejectBecause(parametric_not_flex_out),
    )

    def convert(self, parametric: Parametric[Input[T]]) -> PCM[...]:
        # Things get a little problematic here.
        # This needs some way to either use a provided converter callable
        # OR to use the Implicit converter.
        # I think the steps there are to move the "logic" of Implicit to
        # the top level, and write overrides for this method.
        # Anyway, once that's done, porting over the main logic from
        # artifact.Output is a cinch.
        # One thing that I realize I should do first is, check how aligned
        # the different implementations of the Artifact protocol end up
        # looking.
        # I may want to look into reducing the degree of runtime
        # polymorphism somehow, although there are some hurdles there.
        # Like the fact that there are three separate implementations on
        # both sides of the interface boundary.

    # Anyway...

    @attrs.frozen
    class AdditionalOutput(Generic[T_contra, T_co]):
        map: Callable[[T_contra], T_co]
        converter: Converter[T_co]

        def convert(
            self, parametric: Parametric[Input[T_contra]]
        ) -> PCM[...]:
            # Map types over the parametric, then pass it to the converter.

    # This raises the question of whether the artifact.Output should derive
    # all of its outputs from the basal combined value, or if it should
    # apply a map to that value, and then additional maps on top of the
    # first.
    # I'm thinking about this, and not coming up with anything compelling,
    # so I think the tiebreaker is ease of implementation, which would mean
    # merging those classes into one and constraining T_contra when using
    # them with artifact.Output

    # ...

    # One possibility is to allow nesting AdditionalOutputs inside each
    # other.
    # This would allow quite a bit of flexibility, while also simplifying
    # the implementation of Output...

I think I should keep on thinking about this. Before I make any changes one way or the other, I should switch topics to the coverage topic, and work on marking up all of the code that I decided to change.

This entry is finished a bit earlier than most of the recent ones, so let's just call it there and let me wind down properly for a change.

Good night.

Diary 2023-01-08

2023-01-08T05:00:00-05:00

Not a great day for accomplishing stuff. I'm going to have to really try to work on my various projects tomorrow.

As for right now, I'm going to get to bed, and maybe take some notes on projects while I'm over there.

Good night.

Diary 2023-01-07

2023-01-07T05:00:00-05:00

Okay, let's see what I did...

I'm letting some ideas for MOTR kind of percolate. I'm also trying to get started on another project, and it's just... Ugh, the writer's block is so intense.

I'm going to have to figure out a way around it, but not right now. Like, tomorrow. I need to wind down for now.

Good night.

Coding 2023-01-06

2023-01-06T05:00:00-05:00

My planned changes to MOTR's top layer are getting a bit more ambitious. Basically, I want to abstract out the logic of "what actual operations get performed on artifact.Output's data".

The basic idea is to make the core logic kind of transducer-y, and have supplementary classes that can add additional behavior for specific output types. This is much cleaner in my head than the current system, which kind of mingles different stages of computation, and gets it all to "work" because the types are constrained. If I can pull out the type-specific stuff and make the remainder generic, then I should be able to ignore "what specific stuff the code does in the main methods", because the methods themselves can't act on that information; they can only pass stuff to helper objects with a matching parametric type.

I'm going to need more time to work through these ideas, but I think this is all going in the right direction. I need to get to bed for now, though.

Good night.

Diary 2023-01-05

2023-01-05T05:00:00-05:00

I was distracted working through the Cracking the Cryptic hunt. That's one puzzle past the intro, and it was, um, tougher. And the one after that... I don't even know. We'll see.

Anyway, nothing more to talk about tonight. I'll get back to work on MOTR when I feel like it.

Good night.

Coding 2023-01-04

2023-01-04T05:00:00-05:00

I have a hypothesis about why InputAccumulator is troublesome to rename. Let's see if it makes sense in text as well as in my head.

The problem is that it's basically a reified right fold (hey, spell check is back! Too bad it doesn't know that "reified" is a word.) Anyway, that's not the whole story, but I don't think the bits I'm leaving out matter, except insofar as they make everything more awkward.

The point is, let's have a look at how a right fold is typed in, say, Haskell. (a -> b -> b) -> b -> [a] -> b The InputAccumulator is like this, except the types are specialized and I pretended that I can have existential types. The "list" basically doesn't exist in any explicit sense, and only implicitly exists in terms of the object that gets built up. So, it's like, let's pass the function and the initial value to foldr, but it's not really foldr. It's like... (a -> b -> b) -> b -> wait actually I don't know if it's possible to express this in Haskell's type system, and I'm fairly sure most people wouldn't want to I guess the closest equivalent would be ((a -> b -> b), b) -> a -> ((a -> b -> b), b) or something, subject to additional constraints that I believe the types involved are expressive enough to not give us for free... I'm not saying that the fact that this concept is awkward in Haskell necessarily means I shouldn't do it in Python the way I'm doing it, but given the way that my hobby Python sometimes ends up looking, it does legitimately raise the question. Because the actual fold stuff is only done in one place in the code, it's not a great burden to rewrite it. (Except that I would also need to surface the possibility of mapping over the final value, which is a thing that I do sometimes because of reasons, but surely there's a more sensible way to do that, as well...)

What this would basically entail would be pulling implementation details of InputAccumulator into artifact.Output, and replacing its methods with some helper functions. This would have some side effects in which error messages make more sense, because the failures are coming from the "correct" class all of a sudden. The main issue then is how to package this up and then unpackage it for the artifact.Output, when there's no longer a single class to work with. This is going to take some more thought.

And right now, it's already way too late.

Good night.

Weekly Roundup 2023-01-03

2023-01-03T05:00:00-05:00

Wednesday: I continued to plan renamings for MOTR.
Thursday: More renaming stuff...
Friday: I had trouble renaming one module, at least temporarily.
Saturday: I sketched out ideas for making one part of MOTR's interface make a little more sense, which unfortunately entails a lot of changes under the hood.
Sunday: I moved on to renaming other things, and had trouble with that again. Maybe I should be approaching this like... if I got rid of this module, what functionality would that impede?
Monday: On the road again...

Next week, I'm going to try to keep up with what I've been doing. Maybe change some things up, maybe not. (Ugh, what did I do to Neovim to break my spellcheck under some circumstances?)

Diary 2023-01-02

2023-01-02T05:00:00-05:00

All right. We did some traveling today, and I am super happy to take that excuse to have a short post. I'm going to work hard to decompress, because, despite my genuine efforts, my sleep health ended up... questionable, over vacation.

I can't think of anything else to mention. I mean, I've been tearing through Cracking The Cryptic's hunt for January, but I assume it's going to get harder now that I'm past the intro.

Anyway, nothing else to waffle about, so let's finish this.

Good night.

Coding 2023-01-01

2023-01-01T05:00:00-05:00

I'm going to write a quick post tonight, and try to finish up with plenty of time until the new year.

I was focusing on renaming the input_accumulator stuff. Once I feel good about those names, I'll put the commentary from the last post into the flex source, and then get to work on updating names.

Right now, I'm thinking that ValueAdaptor gets renamed to Selector, and the InputAccumulator protocol maybe gets renamed to MultiSelector. I may also consider collapsing the implementations back into a single class that handles mapping somehow, I'm not sure. MultiSelector is probably not quite right. The idea is that a Selector combines a select.Label and a function to map the wrapped value to a path segment, and the other class is responsible for folding the segments together with an initial value. So, ideally, there should be something more descriptive of that than MultiSelector or Selectors. CombinedSelectors? SelectorCombiner?

I hope it's clear why I think this needs some time to bake.

Anyway, I'm about done with this for now. I'll think about it more next year.

Good night.

Coding 2022-12-31

2022-12-31T05:00:00-05:00

All right, starting the entry a little early today, because I just came up with a breakthrough, and I need to document it to get it all the way fleshed out.

Basically, I've got a provisional name replacement for flex: artifact. Once I had that, I started looking over the various bits defined in the module, and I realized something about some of the helper classes I wrote: they're not so much misnamed, as they maybe shouldn't exist at all.

This turns out to be trickier than I thought it was before I started typing. Let's have a look at two Part definitions that illustrate what I need to work through.

Here's how they look currently:

HTML_DIR = _parametric_command.Extra(
    _flex.FlexOut[str](
        _reports.accumulator("coverage", suffix="index.html"),
        str,
        _parametric.no_parameters(("mypy",)),
        HTML_LABEL,
        _flex.ArgParent("--html-report"),
    ),
)

...

JUNIT_XML = _parametric_command.Argument(
    _flex.FlexOut[pathlib.Path](
        _reports.accumulator(suffix="junit.xml"),
        pathlib.Path,
        _parametric.no_parameters(()),
        JUNIT_LABEL,
        _flex.ExtraParent(),
    ),
    "--junit-xml",
)

And then let's update the names in those and make a few other tweaks:

HTML_DIR = _parametric_command.Implicit(
    _artifact.Output(
        _reports.accumulator("coverage", suffix="index.html"),
        str,
        _parametric.no_parameters(("mypy",)),
        HTML_LABEL,
        _artifact.ArgParent("--html-report"),
    ),
)

...

JUNIT_XML = _parametric_command.Argument(
    _artifact.Output(
        _reports.accumulator(suffix="junit.xml"),
        pathlib.Path,
        _parametric.no_parameters(()),
        JUNIT_LABEL,
        _artifact.ImplicitParent(),
    ),
    "--junit-xml",
)

Some things that come to mind are relatively minor, such as that Argument should probably have an alternative constructor called something like prefixed.

But the big issue that I need to figure out is those ArgParent and ImplicitParent classes. What I hope to do is to essentially turn the HTML_DIR definition "inside-out", so that the description of how the command is actually constructed lives at the top level. I think technically this changes the semantics because the Artifact instance has less control over how it's "rendered", but I also think that doesn't matter, because the Output instance should only appear in one context anyway.

(Also of note, there's another Artifact implementation that corresponds to output, and it should also have access to whatever the main one does in this area; currently, it does not.)

Comparing these usages, the HTML_DIR is saying "if we ensure that this directory exists, we can predict the file that will be written to it". The JUNIT_XML is saying "let's write a file to this path, and we had better make sure the parent exists". In the former case, the child is implicit in the constructed command. In the latter case, the parent is implicit.

Let's try changing things up modestly.

HTML_DIR = _parametric_command.Argument.prefixed(
    "--html-report"
    _artifact.Output(
        _reports.accumulator("coverage", suffix=""),
        str,
        _parametric.no_parameters(("mypy",)),
        HTML_LABEL,
        _artifact.ImplicitChild("index.html"),
    ),
)

...

JUNIT_XML = _parametric_command.Argument.prefixed(
    "--junit-xml",
    _artifact.Output(
        _reports.accumulator(suffix="junit.xml"),
        pathlib.Path,
        _parametric.no_parameters(()),
        JUNIT_LABEL,
        _artifact.ImplicitParent(),
    ),
)

The relevant changes to the interface look really small from this perspective, but they represent a pretty involved change to the implementation. Let's skip over the prefixed stuff, because that's trivial.

Including line-numbers for some reference to myself later...

It's hard to say how these changes should relate to lines 265-7, because the map fields aren't typically doing anything interesting. When we get to 268, there's the Input creation, which needs to be modified in the case of an ImplicitChild, but left alone if there is nothing or an ImplicitParent. The expression from 269-78 would similarly need to use the possibly-updated value. Lines 280-2 apply the outer specification, so should use the parametric from before the ImplicitChild is applied. In the event that ImplicitParent is passed, the block starting at line 284 should be entered, but the method calls to the object can be inlined.

Let's now have a look at OutputFromInput. The current implementation is... aggressively golfed in terms of pure statement count, so it's possible I'll need to break it up if I want to replicate this logic. OutputFromInput.convert starts by pulling the Parametric from the wrapped input. This should correspond to line 264. Note that the OutputFromInput expects the Parametric to have been updated beforehand. I might need to change this, because I can't use these new Implicit ideas if I don't have a PathStr to apply them to, which is not guaranteed. Alternatively, the OutputFromInput could require the caller to pass a PathStr, and then have a separate map step... So, assume the wrapped Input must have a compatible type, and then operate on that in the same general way as Output. It looks like 265 onward could work the same, and maybe be a common function. Question: should OutputFromInput accept the injected values? I forget why I had it discard that argument... For what it's worth, it also happens with Input. I'm also looking at the way I'm putting this together and thinking it it would make sense to just move the inject_registry calls off of the "accumulator" classes, to reduce the API surface. However, doing that might break things... Anyway, I believe that OutputFromInput would need to keep some of the logic that's in Input. I think the logic common to OutputFromInput and Output is from 176-86, which replaces lines 118-20 in Input.

I think this is all of the information I need to fix this stuff up. As of this writing, there's time tonight for me to attempt this, but I'd rather focus on continuing to plan.

First thing I'd like to try: removing the inject_registry wrappers and try doing the injection after the parametric is extracted. I kind of want to try that before any more planning, then get into planning seriously.

It's later now. I'm going to try a quick rewrite and see what breaks, if anything.

All right, it looks like that was clean.

I'm not going to put in much more work on this tonight, so let's switch to seeing which files I want to hit next.

validators: Looks good to me.
command: Needs the extra -> implicit rewrite; base_cmd can probably be deleted and noted in case I decide I actually need it; Metadata should maybe be called Command?
input_accumulator: Needs a few passes to get it to make sense.
installer: Needs to be split into two or three modules, which can then be evaluated individually.
label: Is totally fine, no need to worry.
parametric: I'm punting on.

I'll try looking at input_accumulator, then installer, then command. For now, though, I'm going to wrap up.

Good night.

Coding 2022-12-30

2022-12-30T05:00:00-05:00

Well, I'm still thinking about what to call the flex classes instead of Flex*, and I feel like I haven't gotten anywhere yet. I'm still looking for some kind of unified metaphor, but given the somewhat odd way they're created to be passed (outputs have to be created from function or class calls, which look near enough the same, but inputs have to be retrieved from a Parametric), I haven't found anything that really ties any of this up with a bow.

I once again let things get too late, so I'm going to just do a short entry.

Good night.

Coding 2022-12-29

2022-12-29T05:00:00-05:00

I got distracted, so this is late again. Let's have a look at some minor tweaks to parameters.

I want to remove some of the repetition of the module names in the class and function names. Now, that could also bleed out into renaming the alias, because it's currently a little Pokémon-speak-y. (Granted, that isn't stopping me in most places, so it's not a high priority.)

Let's have a look at the generate_parameters function, though. That takes a set of labels, a, let's call it constants for now, and optionally a set of parameters to augment. The result is an iterator over a set of updated parameters objects.

I don't think there's another meaningful way to iterate over parameters objects, so the current way the name works is

fine.

But I wonder if there's a more explicit way to describe what it does.

Like the point is, you pass in a set of labels to iterate over, the constants to draw the values from, and an existing set of selected variables...

In some sense the point of all of these data structures is that they represent selections out of multiple possible values. So, what if we had select.Label, select.Labels, select.Selections, select.SELECTIONS, and select.select? I'll let those ideas stand for now, and move on to other things for the moment.

On the registry/constants side of things, the big change I'd make would be to re-export Label to provide a module interface that's consistent with parameters/select/whatever.

Next up: whether to make any big changes to flex. I feel like it does need some changes, but I'm not sure what those changes look like. One thing of note is that it's not a run-time dependency of parametric, so it's free to draw in many other dependencies...

validators
command
input_accumulator
installer
parametric
cmd
mkdir
parameters/variables/select/whatever
registry/constants/whatever
facts (requirements)
io

Let's ignore cmd, mkdir, facts, and io off the bat, because they're in more core positions. It's a bit much to go into the remainder of those, so let's focus on flex. The basic purpose of the flex classes is to interact with the parameters/variables/select systems to make sure that a given output file is specified once per command and is different for each set of selections used to generate a command, and that related inputs can be passed multiple times to a single command if desired. There are two main classes there, and a bunch of auxiliary classes. Let's see what's needed to instantiate these main classes:

FlexOut needs:
- a "PathStrAccumulator, which is basically a way to take a base path, and update it according to a value derived from a selection.
- a mapping function that takes the final path from the above and converts it to the desired type.
- a parametric that produces the output names; this field is both poorly named, and should be made optional.
- the label corresponding to the FlexIn that gets created; the caller is currently responsible for creating this, but should not be.
- The helper for potentially requiring a parent directory to exist; this has to go somewhere, and I'm ambivalent about its current status.
FlexIn needs:
- a parametric that produces an Input wrapped around the ultimately-desired type.
- an ordered collection of ValueAdaptors, which are what's responsible for deriving Paths from selections to pass into the accumulator mentioned above.
- An additional field that I don't think callers are supposed to care about. Since FlexIn objects don't seem to be instantiated outside of the flex module, this seems like a safe assumption.

This stuff is all kind of named after how it behaves in the context of multiple configurations. It's "flexible" because you can define a command-line argument once, and it will automatically adjust to the pool of parameters it's supposed to select from.

As such, this is a bad name, because what most people will care about is how it behaves in the context of a single configuration. In a single configuration, these classes need to be produced or obtained in various ways to pass into helper functions that create a Part for a parametric command.

I'm going to have to think about this. For now, it's late.

Good night.

Coding 2022-12-28

2022-12-28T05:00:00-05:00

All right, I'm going to try to review the names in my new modules. So, starting in parametric...

label
parameters
registry
typed_mapping
flex

I'm feeling pretty good about label, so let's start by considering typed_mapping. The basic concept of the "typed mapping" is really more like it's a function that conforms to some protocol, and the function can be updated to produce new functions. Something about functions, maps, finite domains...

Honestly, FunctionOnFiniteSet seems reasonable, since the modules and classes are intended to be used with aliases, so there's no concern about length.

So, next up, registry and parameters. These need to be on the short side. One is a function from a typed label to a value of that type. The other is a function from a typed label of a collection of objects to a member of a corresponding collection. I'm a little bit wondering if I want to change them back to objects and items? My problem with that is, I don't think it really communicates the intended readings of "any instance whatsoever" and "an item of a collection".

I could have constants instead of registry, maybe?

I got distracted so it's getting late, but let's leave that there for now. I'll mull this over and try to expand things out.

Good night.

Weekly Roundup 2022-12-27

2022-12-27T05:00:00-05:00

Wednesday: I audited MOTR's dependencies, and put together some general plans for handling updates to them.
Thursday: I reviewed the lines of missed coverage in MOTR.
Friday: I put in some work improving the test coverage.
Saturday: I ended up derailed into trying to remove isinstance checks.
Sunday: Some of the isinstance checks came out easily, and some...
Monday: I continued to try to pull out the isinstance checks, and it continued to be a pain.

Next week, I've got a few more checks I want to pull out, and then I can get back to addressing coverage. I actually finally figured out the """right""" way to remove the ones that were giving me trouble, which may have ended up doing some implicit casts, unfortunately, but oh well. Hopefully, the type annotations are all honest...

Coding 2022-12-26

2022-12-26T05:00:00-05:00

I had a good day today, and got some work done on MOTR, but right now, it feels like Mypy got me a reasonable distance and fixed some bugs, and then it, like, betrayed me.

I was working on replacing isinstance checks inside one module, where part of the implementation was stubbed out because nothing needed that part yet. I was having some persistent troubles writing a function to deal with the stubbed-out code, until I worked out that I was getting type errors when I implemented the stubs, because the type signatures on the stub were wrong, and the stub was papering over that.

Once I'd addressed that, I would have liked Mypy to accept all of the code, but it's rejecting something about the remaining code, and I'm unclear about how to fix it.

I'm going to keep on trying to make sense of the errors I'm getting, and also I'm going to look into whether it's possible to regain some conciseness, after my attempt to remove some nested conditionals resulted in a lot of method signatures.

For now, I should wrap things up, and I might pick at these errors a bit more, even if I probably shouldn't at the moment.

Good night.

Coding 2022-12-25

2022-12-25T05:00:00-05:00

Dealing with these isinstance calls is hairier than I wanted it to be. I took care of some of them earlier today, and had to reconsider my approach for others later. I haven't tried to fix those yet, maybe tomorrow, maybe next week.

In any case, I did replace some imprecise types with mildly more specific ones, and pulled the relevant logic out of the function and stuck it at the top level. That should open up some similar refactorings later, to help flatten out some methods even more.

At the end of the day, it is the end of the day, and I didn't take enough time to write this post up beforehand, so I'm going to cut this off abruptly now.

Good night.

Coding 2022-12-24

2022-12-24T05:00:00-05:00

I had a look at the missing coverage in command_builder.py, and I concluded that, while it should be relatively straightforward to throw together a test at some level of abstraction, I feel like there should be some way to encode the checks that aren't being hit into the type system and remove them from the runtime code.

So, what would that look like:

Create a new typevar for representing the module_args value.
This propagates to the usages of the CommandBuilder in parametric_command, and secondarily to the client-style modules.
The immediate change would be through the addition of a new parameter to ParametricCommandMeta.
One question this raises is whether commands that accept a module argument should be required to be passed one, but it should be safe to err on the side of strictness to start with.
The key point is that the module_or_adaptor field would also need to become parameterized, and I haven't yet thought of anything that seems like a good way to do that. A way to do that is to create a parameterized protocol, and two classes that implement it, somehow scoping the implementation of each to a different value of the variable (probably by using self annotations of the protocol? Does that even work?), which should then localize the with_module calls to the paths where it's statically asserted to succeed.

Thinking about what would improve the codebase, I think the thing to do is to try to eliminate the isinstance checks that have test coverage. That makes the typing a bit easier to reason about, and also eliminates some conditionals, so the behavior of the code should be less squirrelly. If all of that works, then I can look into propagating the type information like I was talking about.

For now, I'm going to wind down, and I'll see what I can do in the next few days.

Good night.

Coding 2022-12-23

2022-12-23T05:00:00-05:00

All right, I pushed up test coverage on MOTR a little bit. I added some error path tests, and improved the errors they're looking for. I deleted some functions that weren't accomplishing anything. It shouldn't be hard to put them back if I discover I need them, but I don't think I'll need them.

I didn't get much done today because of travel; we'll see how I do in the next few days, but I hope to cut down on the coverage misses, then work on renaming and reorganizing, then rework some of the high-level usages, and expand the highest-level wrappers until it's ready for a release.

I don't know how much of that I can get down in the next week, or in the next month. I'm just going to have to take it a bit at a time.

Good night.

Coding 2022-12-22

2022-12-22T05:00:00-05:00

After I thought through those changes yesterday, I decided to prioritize increasing coverage and other improvements for now.

One of the "other improvements" centers around test execution time. I hadn't realized it before, but the test execution is not just taken up by the type tests (which I'll address by splitting the relevant module out into its own set of packages). There are also tests of the customized Build task (which should maybe be called PythonBuild?). These tests take up about half of the execution time, and the type tests take up basically the other half. I can probably speed up the build tests by mocking the cmd module so the Cmd class doesn't actually hit the filesystem. I'll look into that later.

Coverage misses review:

parametric_command is missing one line. That line looks like it could be factored out, but I don't think it's worth it. The required test is to create two ValueAdaptor objects with the same Label, and put them on the same ParametricCommandMeta object, then have a ParametricCommand attempt to build() given that metadata.
python_helpers is missing one line. I think this line will be relevant for running coverage.py. Perhaps the thing to do is to make stub implementations for coverage, and have tests that just import those modules.
input_accumulator is missing one line of coverage, and I'm not comfortable trying to add it until I've audited the relevant names, because the data flow around that line is confusing, and it's more or less my fault.
flex is missing three lines of coverage. One of them appears to be unnecessary and can probably be deleted. One of them can probably be deleted with some care. The remaining line requires a simple dummy test to cover.
command_builder is missing one line, that requires a simple dummy test to cover.
build is missing two lines, which will be much easier to cover if I figure out the mocking I want to use.
installer is missing two lines, which can be tested relatively easily.
pip is missing six lines, of which five can be tested trivially, and one needs a little more thought to test properly.
parametric is missing twelve lines, and probably needs a thorough review to plan tests for them.
command is missing seven lines. I can reduce or eliminate six of them with some careful rewrites, and the remaining line... I honestly think it's best to just delete it, because it doesn't really look like it would pull its weight if I did use it.
not_output is missing four lines, and those lines are easy to cover, but I'd like to consider, later, what it would take to hit those lines in an integrated sense.
validators is missing five lines. Many of the tests above would hit one of them, and I'm interested in writing tests for the other four, just because it'd be a nasty shock if someone were somehow debugging MOTR interactively, and suddenly something's repr was broken.

All right, let's call this here. I'll be on the road tomorrow, so I don't know if I'll be up for writing any of those tests, but hopefully over the next week...

Good night.

Coding 2022-12-21

2022-12-21T05:00:00-05:00

I was thinking about working on MOTR, and I realized that, before I do anything else, I want to audit my dependencies to check for urgent updates. I know Trio has some stuff coming down the pipe that I need to be ready for. And I can make some improvements to how I'm requiring Cement if I use a newer version. Here are MOTR's direct dependencies, and notes on them:

attrs: has a new interface that I would like to switch to at some point.
blessed: It looks like nothing breaking is happening. I'll have to revisit it when I look at some issues with the templates.
cement: newer versions provide a nicer way to pull in the following dependencies:
- colorlog
- jinja2
- pyyaml: I want to figure out what it takes to drop this dependency, because it's for reading configuration files, and I don't... I'm not... You can't fit a motrfile in yaml.
pyrsistent: I think I want to cut down on the usage of this slightly, because I'm fairly sure I'm not playing to the strengths of PVector or PSet.
tqdm: I have some ideas for trying to make the progress bar a little nicer, but I'm going to hold off on trying them out for now.
trio: Some of the interfaces I'm using are changing, and also there are some interactions with Cement that I want to address.
virtualenv
wheel-filename

Let's have a quick look at the Trio changes.

The upcoming change is to trio.MultiError creation, or lack thereof. In order to handle everything properly, I need to:

add strict_exception_groups=True to the trio.run call
replace the contextlib.suppress(_exc.MOTRTaskError) with, um, exceptiongroup.catch({_exc.MOTRTaskError: noop}), or something?
Figure out what Cement does when it gets a BaseExceptionGroup
Fix any tests that this breaks. Like, can I write catch handlers that will preferentially reraise any error besides the expected one, and reraise the expected error type if there are no others? Is pytest getting changes to handle exception groups? I don't think these off-the-cuff ideas are quite enough...

The other Trio-related change I want to make is to stop telling Cement to handle SIGINT. This requires getting a small number of changes just right, which shouldn't be hard, but it took a bunch of effort to verify that this was the case. And I'm still not totally sure.

Anyway, I'll look into making those changes later. For now, I should wind down.

Good night.

Weekly Roundup 2022-12-20

2022-12-20T05:00:00-05:00

Wednesday: I carefully traced the data flow required to instantiate an environment in MOTR's high-level interface, because it's really not obvious how it all fits together.
Thursday: With that information at hand, I was able to improve the quality of the new tests in MOTR.
Friday: I reviewed how some of the code paths I want to test are meant to be invoked, and planned some stuff out.
Saturday: We watched the MST3K season finale. That was a lot.
Sunday: I got sidetracked by Jerma's holiday stream(s), so, no normal post.
Monday: I tossed together the tests, and got MOTR up above 97% coverage. I concluded that the names at the high level really need to change.

Next week, I'm going to relax for a few days, try to work on MOTR, relax a bit more, then work a bit more.

Coding 2022-12-19

2022-12-19T05:00:00-05:00

I did a bunch of other stuff today, but at long last, I managed to grind out the tests to increase coverage for the flex.py module. In the process, I established that "bundle related arguments together" seems to work fine, but also that a bunch of my names don't get any more intuitive the more I stare at them, so I really need to look into changing them around.

I guess I'll try to take notes on that starting some time next week. Anyway, not going to dwell on this stuff right now. Best get things wrapped up.

Good night.

Diary 2022-12-18

2022-12-18T05:00:00-05:00

Hm, I didn't anticipate how things were going to go in terms of stuff to do today, so I don't have anything much to write about.

I did do some reorganization in a side project, and messed around in a different side project, but nothing really worth writing about yet.

I'm going to try to wrap things up for now, and actually get stuff done for MOTR tomorrow.

Good night.

Diary 2022-12-17

2022-12-17T05:00:00-05:00

I'm not sure I can process anything after the MST3K season finale. That movie was very, very little, until it was so, so much, and then it went back to being nothing. Like the movie blew its entire emotion budget on one scene at the end? Bizarre, on so many levels.

Anyway, let's see what I'm up for doing tomorrow; I really need to wrap up right now.

Good night.

Coding 2022-12-16

2022-12-16T05:00:00-05:00

The next step for increasing test coverage for MOTR, is to refresh my memory. I see that the next stuff I want to exercise is currently, in existing code, kind of just tossing around FlexIn objects in a way that isn't quite satisfying, according to my gut reaction.

Like, this shouldn't be relevant to the tests, but it feels a little weird to have the same FlexIn object appear twice in the expressions going into the command... Like... the FlexIn and its derived LessFlexOut should always be paired, but I guess I don't have enough examples to decide on how to do it generally. Although, it doesn't have to be general. It could just as well be a function that returns a sequence of Parts, and I just call it like .add_parts(*func(...)).

When I have time to work on the tests, I'll have to experiment with making interfaces like that, and update the existing code if the result is satisfying.

Anyway, I let things go way too late again, so I have to cut this off here. I'm not sure I'll get anything done tomorrow, so let's see how the weekend goes.

Good night.

Coding 2022-12-15

2022-12-15T05:00:00-05:00

All right, I've properly juiced up the test that I wrote. It turned out to, um, not need the helper functions I wrote, because those make sense in the context of MOTR's python- and pip- related hooks, and when I'm literally faking every part of the commands, it doesn't really make a difference.

My goal now is to extend this test to run through more elaborate logic, feeding command outputs to inputs in a secondary command, and mapping those inputs back to outputs. I'm not going to try to do that now; I finished this up a little earlier than usual, and I'd rather take the extra time than extend this work out tonight.

Good night.

Coding 2022-12-14

2022-12-14T05:00:00-05:00

All right, let's take a look at MOTR's code...

My goal here is to trace things back from the flake8 test. Let's see how this breaks down...

test_flake8.make_changes defines a function called make, which takes two arguments: parametric and package_data.
- package_data is passed into api.cli.build.build to return Facts[Objects], and inject information about which Python versions to use for flake8. The package_data comes from api.package.combine_packages. This is a PythonPackages object, and the relevant behavior partially lives in the constants defined in that package, and partly in api.cli.build.build(), which puts the necessary values under the necessary keys. TODO: make a helper function in package for carrying out the changes, rather than having them done in the return statement of api.cli.build.build().
- parametric pulls in the information from the modified Objects, and the argument further down is from api.cli.flake8.BASE.add_parts(...).build(). This add_parts() return value is a ParametricCommand. Its build() method calls the command_builder.CommandBuilder.facts() method from the _build() method, which is really complicated, but I think the important thing is the ParametricCommandMeta that had with_adaptor() called on it.

Okay, I'm lost again. Let's check out the facts() method? Oh, wow, my names for these things got janked. Basically, at some point, parametric information about how to construct the specific strain of environment required for the operation is injected into the ParametricCommand via with_adaptor(), and the build() method on that object retrieves it, and passes it to facts(), which combines it with the installer_registry data to pass it to installer.install(). The installer_registry is on the command.Metadata, and is, in these specific cases, populated from the PythonCmd helper.

So, for a synthetic version to work, I need to create a command.Metadata that has the desired installer data stored under a known key. In addition, I need a versions adaptor wrapped around a known key, which needs to be explicitly passed in, associated with a PVector[installer.PathWith[EnvironmentArgs]].

I apologize for how completely and thoroughly opaque that was, but I needed to go over this really thoroughly in order to remind myself where everything actually gets instantiated.

Maybe tomorrow, I can actually do something with all of this. For now, I'm done.

Good night.

Weekly Roundup 2022-12-13

2022-12-13T05:00:00-05:00

Wednesday: I tried to figure out the proper interface for a test helper function.
Thursday: I got the proper interface, by splitting the function in two. I also made a slight organizational change to the tests directory.
Friday: I confirmed that the helper functions worked, and cleaned up conftest.py a little.
Saturday: I realized that I've forgotten how (and whether) the various interfaces required for using a high-level wrapper around a Python script actually get invoked.
Sunday: Travel out. Fried.
Monday: Travel back. Fried.

Next week, I'm going to very methodically document the flow of data through my tests, so I can figure out how this code is currently "supposed to be written", and whether I want to change that. And other stuff.

Diary 2022-12-12

2022-12-12T05:00:00-05:00

Okay, we traveled again today, so I just didn't plan to do much to write about. My plan for the coming week is to nail down whether and how the current high-level tests for MOTR are creating correct Compendium objects, and to pick up some side stuff that I got unblocked on a technical level yesterday.

For now, I'm going to wind down and get ready for bed.

Good night.

Diary 2022-12-11

2022-12-11T05:00:00-05:00

We traveled today, and I spaced out.

On the plus side, I did manage to finally fix some abstruse technical issues in a side project, so now I can get back to working on that.

Besides that, I've been working on the latest puzzle hunt from the Cracking the Cryptic patreon. I've sort of solved it. I've got the last puzzle solved, but I keep messing up the second-to-last puzzle, which is supposed to be a prerequisite for the last puzzle. I should hopefully get things done reasonably within a week. Anyway, I let things go way too late and I desperately need to wrap up.

Good night.

Coding 2022-12-10

2022-12-10T05:00:00-05:00

All right, let's see what I was doing with these high levels of abstractions...

The "pattern" for interacting with with the code seems to be as follows:

Given the label associated with the versions, also have a PVector[PathWith[EnvironmentArgs]].
Also, construct a Parametric[Objects] that contains the proper data to run the commands, which is... um...

Okay, so this all made sense at the time. I think. I've got a fake Executable which contains a label, which needs to be used as an index in the installer registry...

You can tell this code isn't ready for anyone else to use, because it kind of seems like it's not ready for me to use it...

Okay, here's the plan. I'm going to take things easy this weekend. Maybe work on something else. Then next week, I'm going to document the data flow very, very carefully, and take notes on any names that could be different, because the fact that so many of these names lack any semantic content doesn't help anything. For now, I need to rest up and get ready to travel.

Good night.

Coding 2022-12-09

2022-12-09T05:00:00-05:00

The new test helper code I added is in good shape now. Running motr on itself results in everything being good except for missing about 50 lines of coverage, out of 2360.

I wired the new helper functions into some existing tests, and the result is... I don't feel strongly, but I'm not changing it back. Now, I just need to look over those old tests, and figure out what I need to get the new tests to exercise more code paths. Basically, to take the existing tests, and copy their logic. This may end up requiring some of the code that I removed in revision 672 because the current low-power tests didn't need it. Which is exactly why I made sure to have a separate commit removing it; this way it's easy to bring it back.

Anyway, that's what I'll try to work on the next chance I have, which, I don't know for sure when that's going to be, but whatever.

Good night.

Coding 2022-12-08

2022-12-08T05:00:00-05:00

Okay, I managed my time poorly once again, so here's what I dashed out quickly, and then I'm going to publish:

I added a __init__.py file to the tests directory so I can do a relative import of the helpers module I just added. Maybe I'll reorder that a little later, but there's no time right now. Inside the helper file, I put two functions. One takes a Facts[Objects] and an Objects, and calls .update() on the Objects that it gets from the Facts[Objects], with the other Objects that was passed. The other function takes a Facts[Objects] and any number of Parametric arguments. It extracts the Objects from the Facts[Objects], then calls .facts() on each Parametric that was passed in.

Together, these sidestep the API design issues I was having by having two smaller functions that do less each than the one large function I was visualizing, and are less awkward to lay out the parameters for, thanks to the level of nesting this adds.

Anyway, I'll actually have a chance to try this stuff out later. For now, I just wanted it put together and ready to think about. No more writing for now.

Good night.

Coding 2022-12-07

2022-12-07T05:00:00-05:00

All right, I ended up spacing out and it's late, so here are some quick notes about the helper functions I want to consolidate:

There is an initial call that produces a Facts[Objects], followed by merging a limited amount of data into the Objects.
Then, for each Parametric passed in, just yield from the Facts, given the Objects.
I was interested in verifying the Objects against the stated requirements of the Parametrics.

I'm not sure how much caching I want to do. I guess, on reflection, I could get away with making this a helper function, but it's a little tricky to visualize the right interface. Like, is def __call__(self, facts: Facts[Objects], overrides: Objects, *parametrics: Parametric[object]) -> Facts[None]: reasonable? I'll need to think about it.

For now, I'll get ready for bed.

Good night.

Weekly Roundup 2022-12-06

2022-12-06T05:00:00-05:00

Wednesday: I wrote down the apparent requirements to make a fake program module in MOTR.
Thursday: I implemented most of those requirements, except for the last one.
Friday: I put together something that may or may not implement the final requirement.
Saturday: We watched Species.
Sunday: I started planning tests for the uncovered code.
Monday: I added the most basic of coverage, and immediately found a bug.

Next week, I'm going to try to improve the quality of the new test in MOTR, and tighten up existing tests.

Coding 2022-12-05

2022-12-05T05:00:00-05:00

I added a small amount of code coverage today, and it immediately revealed a bug that had eluded all other checks. This has happened a bunch before. Whenever I write tests for some squirrelly bit of code, it seems to always turn out it was accidentally making bad type assumptions, or has a critical bit of logic inverted, or whatever.

I've read stuff online talking about how full code coverage isn't valuable, but I'd really rather exercise these weird code paths before I ship something. Maybe other people can write this kind of code with no bugs. If so, good for them, and when they're working on their code bases, they're free to apply standards to their work that match up with that track record. For my hobby projects, I'll be applying standards that apply to my track record with these hobby projects, and that means 100% coverage as table stakes.

Now, as it turns out, the test I wrote, that one that found the bug that I had to fix, is actually kind of on the weak side of things, and I'm going to need to extend things in several ways, some of which will increase coverage, and some of which won't, at least outside of the sense of "tests should have 100% coverage, so adding a new test should move the coverage percentage closer to 100%".

I need to test the EnvVar-related code paths for the other Flex implementations besides FlexOut.
I need to generate a proper Compendium from test data.

I may prioritize the latter, just because it'll make existing and new tests much stronger.

Actually, that's a good thing for me to consider while I work on the tests: should it always be the case that a valid ParametricCommand's build() method should produce a Parametric that, when passed all required (and possibly some additional, unnecessary) data to its facts() method, produces a Facts that, when passed to api.build(), produces a Compendium without erroring on any step? I think that's a reasonable correctness criterion, so, either later tonight, or sometime next week, I'll see about writing a helper to handle that.

The helper would have to do stuff like validate that all necessary data is passed, because right now, the code doesn't really establish that. (It would also need to precompute some of the facts to actually get that far, but, um, oh well.)

Anyway, I'm going to wrap this up and post it a little early, so I can relax later tonight.

Good night.

Coding 2022-12-04

2022-12-04T05:00:00-05:00

Well, it turns out I was taking things easy again today, so let's see about the kinds of things I need to be able to run the tests I want to write. Starting with the attempts to just exercise the code.

I need to create a label object, and from there, the first thing I want is to create something like parametric_command.EnvVar(flex.FlexOut(reports.accumulator(suffix="test.txt"), lambda x: x, parametric.no_parameters(()), label), "TEST_VAR"). Let's see if it likes having that code in there, not executed yet.

I could try to adapt the existing tests into versions that interact with this new stub code, but I don't feel like pushing myself further right now, and I'm going to hold on to feeling satisfied with the amount of work that I happen to have done at any given time.

So, I'm done for now, and we'll see what I manage tomorrow.

Good night.

Diary 2022-12-03

2022-12-03T05:00:00-05:00

Okay, I didn't get anything done today that I want to write about. We saw a movie. It was silly.

I'm spacing out, and I don't have anything else to say.

Good night.

Coding 2022-12-02

2022-12-02T05:00:00-05:00

So, I might have been kind of out of it again today. The air is really harsh and makes it hard to breathe and stuff like that. Anyway, let's crack open my... unreleased legacy code?

The core question here is, what do the first two arguments to CommandBuilder need to be? The first argument is an Executable, which includes a script name. The second argument is a command.Metadata, and it is optional, so I might be able to get away with not passing anything.

...

Sadly, maybe not. The existing examples pass installer information in the Metadata instance, which... Wait... They're only passing that because the pip command has to actually accomplish something. The installer information doesn't establish any task dependency relationships, so it should be safe to leave the installer be. So, let's see what I think will work...

Okay, I have some syntactically valid Python code here. The goal now is to exercise unused code paths in my new wrapper code, and hopefully figure out the correctness criteria for writing the tests, because I confused myself badly the last time I tried to work that out.

I think the fundamental goal is: no matter what combination of parametric labels and parametric combinators, I'm never trying to put two different values into the same environment variable.

Anyway, we'll see when I'm able to put all of that together. For now, I'm going to wind down again.

Good night.

Coding 2022-12-01

2022-12-01T05:00:00-05:00

I didn't really focus on MOTR today, so I'm going to try to make some quick work around what I worked out yesterday.

...

All right, that's enough for now. I've gotten nearly everything I wrote up yesterday sketched out, and what remains is... unintuitive. There's a bunch of ... stuff ... that the wrappers I've written up all use, but the helper code is ostensibly Python-specific. So, what I need to do is, when I'm feeling better and more focused, is to re-work-out the types and values involved in constructing the arguments to these functions.

For now, I'm going to wind down again.

Good night.

Coding 2022-11-30

2022-11-30T05:00:00-05:00

Okay, it's late, and I'm trying to get up to speed with how I put things together in MOTR's wrappers. I'm going to list the stuff that looks necessary to see if it gets everything I need to know, into my head.

A subclass of Program.
An adaptor around a ParameterLabel[PathWith[SomeEnvironmentArgs]].
A Label[PathWith[SomeInstallerArgs]].
Specify the type of entry point.
An alias to CommandBuilder[EntryType, ProgramSubclass, SomeInstallerArgs].
An alias to OptionBuilder[ProgramSubclass]
An executable wrapper around the installer args label, the command name, and an instance of the entry point type.
The final absolutely-required bit is kind of hard to describe; it pulls together most of the above.

Things I'm noticing:

There are some names that I didn't update yet.
Others of the names don't yet have something to update to, but they need to be changed. There are a lot of methods that are (obliquely) named after internal concepts, and it's not obvious from these names why you'd want to ever call the methods.

Tomorrow, I'm either going to try to fill in dummy versions of that stuff in the test code, or do something else entirely. We'll see then. For now...

Good night.

Weekly Roundup 2022-11-29

2022-11-29T05:00:00-05:00

Wednesday: I didn't get much done because we were on the road.
Thursday: I roughed out some code that I'm really not sure whether it would have worked.
Friday: I figured out that there were other issues with the code, unrelated to whether it would actually work, so I scaled back my changes. I started updating the code base...
Saturday: ...and finished up those updates, had a slightly tricky merge, then started planning where to go next.
Sunday: I started working on those plans, and quickly realized that I'm going to need to rework the public interface of MOTR to make more sense.
Monday: I didn't get much done because we were on the road.

Next week, I want to consider making a few more changes to MOTR's code, but maybe I'll just move ahead with writing the tests.

Diary 2022-11-28

2022-11-28T05:00:00-05:00

We traveled some more today, so I'm not going to try to spin things up for MOTR.

What happened today... We watched the Roger Corman House of Usher. It was pretty good, especially by the standards of Roger Corman Poe movies.

Let's see... Nothing else to report.

Good night.

Coding 2022-11-27

2022-11-27T05:00:00-05:00

I started putting together dummy instances of the various protocols that need to be implemented for a new MOTR... um... middle-end? There are two directions to go from here:

Let's see what's absolutely necessary from looking at the existing modules for wrapping third-party utilities.
If it's a little inconvenient and fiddly for me to implement this stuff in an internal testing library, it'll be so much worse for hypothetical third-party maintainers.

Now, I'm not imagining that anyone else is going to want to use MOTR—certainly not in its current state, and, in terms of simple probability, probably not after I've tried to polish it up some—but the entire point of MOTR is for me to get deep into its guts for the past few years or whatever, so I can avoid going to the same effort for other projects.

A quick review of the general user profiles:

Someone who doesn't care about any of the internals; some other project is using MOTR, and they rely on the maintainer to work out how it should all fit together, and document any quirks of usage.
A project maintainer, who should be able to write a concise motrfile from built-in and third-party wrappers, which provide well-defined extension points for things like plugin arguments.
A wrapper developer, who should be able to write straightforward wrappers around third-party utilities. (The current state of the wrappers shows the "straightforward" part to be, um, deeply aspirational.)
Me, who is secretly all of the others as well.

The problem with the current api module is that it's approximately the motrfile helper code that you need if the wrapper code does not exist, which it didn't at the time, but now it does. Mostly.

Anyway, I think in order to move forward, I'm going to want some kind of separate modules for the two use cases. Ideally, the built-in wrappers can simply use the module for third-party wrappers.

Something like...

Here is a module that allows you to define environments and installers
Here is a module that allows you to write wrappers around Python scripts
Here is a module that allows you to write a motrfile using wrappers

And the existing monolithic folder that contains basically all of that needs to be split up.

I'll look into that after I've gotten coverage properly up. For now, I want to finish this entry before it gets really late.

Good night.

Coding 2022-11-26

2022-11-26T05:00:00-05:00

Okay, I got some solid work in on MOTR today. I finished up the typing issues, fixed some minor formatting issues, and had no test issues, which is a good sign, because I shouldn't have been changing any behavior.

There were some issues with the merges.

The first merge had the issue that it was not a merge, because I hadn't diverged from an ancestor, so instead I had to, from the changed topic, mark the current topic as the ancestor topic, and add an empty commit. I feel like there should be a better way to do that, in terms of ergonomics, and there probably is; I just didn't ask.
The second merge was more of a straightforward "there are pervasive changes in the code that are right next to each other, and the diff algorithm decided to be cautious".

Anyway, now I'm feeling good to add tests around this code, now that the types for it make a bit more sense. To review what I need to start with:

InstallerArgs instance
EnvironmentArgs instance

There are some other things, but I don't want to think too hard right now. Once I've had some more sleep, I can review the relevant code, now that I've messed with it, and figure out the right way to access the relevant classes and functions.

For now, I need to get ready for bed.

Good night.

Coding 2022-11-25

2022-11-25T05:00:00-05:00

Okay, I wasn't able to focus all that well today, and I'm going to blame Thanksgiving for that. Now, let's see what I've got...

I thought some about that snippet I posted yesterday, and tried to fix it up a bit.

class MultiInstaller(Protocol[TInstaller_contra]):

    def __call__(self: MultiInstaller[TInstaller], __key: Type[TInstaller]) -> TInstaller:
        ...

All right, let's see what it takes to make this happen...

...

Well, I'm trying to do sensible things here, but there are limits. I'm not totally sure if this is all going to work, but if it does, I'm going to need to rethink some of my decisions.

Either I need to rethink my earlier decisions, or I need to rein in the scope of my changes, which is... not a huge problem, because that means deleting a bunch of code that I don't trust. Actually, let's try reining stuff in now...

...

Changes are rippling outward. Either this will end up completely unworkable, or it will be poised to make a lot more sense. We'll see...

Anyway, it's too late.

Good night.

Coding 2022-11-24

2022-11-24T05:00:00-05:00

So my focus today was... bad.

Let's see what I can do for MOTR, quickly.

So, I took some notes earlier, and there's one idea that they just kind of glossed over, and there's a chance the quick-and-easy implementation of this idea would not work...

Basically, I want a typed mapping Protocol argument that looks something like

class MultiInstaller(Protocol[TInstaller_contra]):

    def __call__(self: MultiInstaller[TInstaller], __key: Label[TInstaller]) -> Installer[TInstaller]:
        ...

This code is pretty fiddly, even if you ignore the fact that it needs weird plugins to work with Mypy. My major task in the coming days is to evaluate whether that code would actually work; it doesn't quite fit with the other usages of TypedMapping, and it could be just different enough to fail typechecking.

If that doesn't work, I'm going to need to work out some of the TypedMapping-like stuff that I want from scratch.

It's kind of a pain, but if I can get this to work, it will make some of the new code both more robust and powerful, and easier to understand, in an area that sorely needs robustness and comprehensibility.

Anyway, it's way late and I need to wrap up.

Good night.

Diary 2022-11-23

2022-11-23T05:00:00-05:00

We traveled more today, and so I'm kind of out of it. On the plus side, I've got some experimental changes to MOTR planned that should address some of the misgivings I have about the current versions of the type hints. Hopefully, I'll be up for trying them out in a side topic tomorrow.

Anyway, I shouldn't be procrastinating like I am, so this post is getting the short end of things, as it were.

Good night.

Weekly Roundup 2022-11-22

2022-11-22T05:00:00-05:00

Wednesday: I made some obvious improvements to MOTR's code.
Thursday: I started to refresh my memory on how some of MOTR's new code holds together. I came up with some unfortunately-complicated ideas about how to make certain areas of the code reusable.
Friday: I came up with some much less complicated ideas, that should suffice for now.
Saturday: After focusing on the improvements side of things, I decided to take a look at the uncovered code in MOTR. In my experience, such code is suspect, because it tends to be the case that I'm not manually exercising uncovered code. Sadly, this part of the code is very elaborate and hard for me to follow without walking myself back through things.
Sunday: I started planning out how to write tests for specific areas of uncovered code.
Monday: I was not up for working on this stuff, so I tried to take things as easy as I could manage, which is... eh, not very, when I only think to check how I'm doing at the end of the day.

Next week, I'm going to be traveling some for the holidays, but hopefully I'll get some time to actually touch the code.

Diary 2022-11-21

2022-11-21T05:00:00-05:00

I could start on the work I set myself for improving MOTR's tests, but I don't feel like it currently. I've got a headache right now, and I just want to lie down and shut out the world for a bit.

I sat around for a bit to see if I'd think of something else to write, and I did not, so I'm going to stop pushing myself for tonight.

Good night.

Coding 2022-11-20

2022-11-20T05:00:00-05:00

Unfortunately, I didn't get started until late today. Fortunately, I have a bit of time. So, I'm going to try to lay out what I need to make a ParametricCommand where I don't need to think about how the pip-related stuff works.

The basic ParametricCommand comes from calling ParametricCommandMeta.with_adaptor(). The argument is a ValueAdaptor[PathWith[EnvironmentArgs]]. The calling object is a ParametricCommandMeta[TEntry, TProgram, TInstallerArgs]. So, to make a version of this with test hooks, I need to pick an Entry type, subclass Program, and create custom implementations of InstallerArgs and EnvironmentArgs. (It seems slightly unfortunate that the types of InstallerArgs and EnvironmentArgs implementations don't interrelate in any sensible way, but I don't think I can deal with that without higher-kinded types or a deep redesign, which may be called for.)

There's a bit more to be done around this stuff, but it should be really easy to make basic implementations, since I don't think the implementations need to "actually do anything".

In any case, I've run out of time for now, so I'm going to wrap up for now, and try to work on this in the coming days and weeks.

Good night.

Coding 2022-11-19

2022-11-19T05:00:00-05:00

Okay, I've got a bit more time. Let's see about improving MOTR.

After doing some planning for rewrites, I decided to take a look at coverage misses, because I'm currently blocking these rewrites on getting the coverage up.

One really prominent set of coverage misses focuses around the EnvVar class, which it looks like I'm currently not using at all. EnvVar is one of three wrapper classes that go around a Flex. These wrapper classes take the value in the Flex, and inject it into the generated command in various places. EnvVar is for injecting a Flex value into an environment variable. I don't know offhand of a program that works that way, but I think I shouldn't discount the possibility.

As far as testing this behavior, I could try to verify the actions of the functions directly, but I should be looking into verifying the design, and I'd like to try and verify the intended higher-level behavior, which is that you can't "double up" on a Flex's value by parameterizing fewer variables than the Flex contains.

Let's see what this looks like...

The fundamental range of behavior is between FlexIn and FlexOut, because LessFlexOut should mostly act like a FlexIn.

FlexIn: here is a specific range of values that something else generated.

FlexOut: here is a means of generating values.

The basic check here is that there is:

Every command gets exactly one value for the EnvVar, and no EnvVar of a FlexOut produces the same value twice.

Thinking about this, I'd kind of rather have some kind of framework for testing this rather than trying to put stuff together ad-hoc. I'm going to have to spend some time thinking about that.

Anyway, it's late.

Good night.

Coding 2022-11-18

2022-11-18T05:00:00-05:00

Time management mess-up write post quick go!

I figured out a less disruptive way to handle "this use case requires reaching deep into a nested data structure to swap out a sentinel value": because the actual value is a Label that corresponds to a structure involving the implementation of a particular interface, I can implement that interface with a class that's not instantiable. I don't know for sure if that's how I want to do this long term, but the resulting code should be very close to what I have now, except now I have proper access to "this complicated nested state with a hole in the middle".

Anyway I have to stop writing now.

Good night.

Coding 2022-11-17

2022-11-17T05:00:00-05:00

Let's see. The quickest way to get plans for updating code is to see what corresponds to the updates on the code I haven't written yet.

Um. I'll try again.

I have three-ish groups of modules that aren't implemented in either topic. These modules have some correspondence with other modules that are implemented, but I would like to rewrite to some degree. I feel like it would be "more efficient" if I did the rewrites before bringing in more code that would need equivalent rewrites.

So.

pyinstrument
limit-coverage
coverage subcommands

I'm going to try putting detailed notes in the Kanban board, but I'll summarize things here as I go.

Two of these modules will work similarly to _api.cli.python. So, refresh my memory about what's in there...

Well, this is a little unpleasant. I look at a helper function with a single statement, and there's just so much to unpack about making the code in question reusable.

Like either I need to break up the data at several levels of abstraction, or I have to create a placeholder that's never supposed to be emitted... If I could mark the placeholder as "not-supposed-to-be-used", maybe I'd feel better...

Why is this so hard? Anyway, I'll have to think about this.

Good night.

Coding 2022-11-16

2022-11-16T05:00:00-05:00

I did some basic cleanup to MOTR's code today. The first thing I did was to remove some test code that wasn't accomplishing anything. The second thing I did was to touch up some of the documentation to remove references to outdated terminology.

Looking over what I have queued up for myself, there are two obvious courses of action:

Triage the majority of modules that I wasn't quite sure what to do with.
Look over the empty files, and figure out what I need to know before I feel comfortable filling them in.

In any case, that's not happening tonight. Maybe later.

Good night.

Weekly Roundup 2022-11-15

2022-11-15T05:00:00-05:00

Wednesday: I started listing files in MOTR that I need to look over.
Thursday: I decided to move those files into a kanban board instead of a blog post. I ended up getting distracted complaining about the various options I found.
Friday: I ended up settling on the option that I had the least complaints about. (Basically, since I'm not planning to share this board, I don't think it should be on Trello taking up space. Much better to take up space on my hard drive, NAS, onsite backups, and offsite backups. (... If you were wondering how I was able to get my laptop working again relatively quickly...))
Saturday: I was stressed out so I tried to take things easy.
Sunday: We traveled, so I tried to take things easy.
Monday: We traveled, so I tried to take things easy.

Next week, I'm going to try to actually work through the kanban board, now that I've loaded every file into it.

Diary 2022-11-14

2022-11-14T05:00:00-05:00

And we're back. Traveling in the opposite direction does not have the opposite effect, unfortunately.

I don't really want to look at my screen, so I'm going to keep the waffling to a minimum. Hopefully I'll be up for writing more in the coming week.

Good night.

Diary 2022-11-13

2022-11-13T05:00:00-05:00

We traveled today. I'm not up for writing stuff.

Yesterday, I made some good progress on the task tracking for MOTR. The big thing I ended up realizing/remembering is that I can't just get the coverage to be all the way up, I also have to actually write a bunch of these modules. Hopefully, I can also delete some of them.

Anyway, I'm going to shut things down for now.

Good night.

Diary 2022-11-12

2022-11-12T05:00:00-05:00

I'm writing this entry early, because I want to be ready to go to bed as soon as possible later tonight. I've been under some stress recently, probably due to the time change, some unusual travel, and a few other factors.

I'd really like to be able to get work done on my various projects, but I can no longer ignore the results of pushing myself to get stuff done. So, my plan for now is to publish this entry, and get any work that I feel up to done, all around dinner, and then.

Just.

Take.

Things.

Easy.

Good night.

Coding 2022-11-11

2022-11-11T05:00:00-05:00

This is quick because I was distracted, but I've added a few more of the files in the project to the board, and one thing I'm noticing about my code is, regardless of whether you believe in self-documenting code, some of the stuff I've written is clearly not.

Like, when these functions have four or five arguments, it should be at least a little clearer what the dang deal is.

I just now got an offline Kanban app working, so we'll see what I think of that going forward.

Anyway, I absolutely have to get ready for bed now, and I'm not sure how much more I'll get done tomorrow or over the weekend, but whatever.

Good night.

Coding 2022-11-10

2022-11-10T05:00:00-05:00

So, I decided to track the files to look at in a kanban board, rather than copy around lists in blog posts. I've already copied the previous post's information into a Trello board, but then I got distracted because, like,

Trello is fine, and it definitely makes sense if I want to share my board, but for something like this, I feel like ideally I'd like something that:

Is offline and saves to a local file that I can back up myself
Gives me all of the features out of the box without trying to upsell me
Successfully launches
Bonus: lets me pull in my boards from Trello (/Zenkit, which appears to have... a bunch of the same boards as my Trello? I completely forget what I was doing, and cannot justify this situation to my satisfaction.)

For some reason, this intersection of requests seems to be overly strict. And that's gotten me distracted.

At some point, I think I need to try to build this myself, if I think it should be possible to write an offline kanban app that actually works instead of crashing and saying vague and mean things about my GPU, or trying to upsell me.

Anyway, I'm going to try and add a few more cards to the Trello, because Trello is fine. I should wrap up this post now.

Good night.

Coding 2022-11-09

2022-11-09T05:00:00-05:00

A bunch of stuff happened today, but it's all wrapped up. Now, I want to do something with MOTR.

I ran the tests again to see if anything had broken in the last few days, and it did. A bunch of the typing tests broke because I had a helper function that, as of today, needs a #type: ignore line on the definition. But, that was simple to take care of once I figured out that the error that was failing every test wasn't coming from every test.

And now, I've got all of this code to look over.

Nothing specific:

The type plugin code
_api/cli_types/command_builder.py
_api/cli_types/entry.py
_api/cli_types/flex.py
_api/cli_types/input_accumulator.py
_api/cli_types/label.py
_api/cli_types/not_output.py

Something specific:

_api/build.py needs a documentation update.
_api/cli_types/command.py has that helper function that I'm not sure whether it should exist.
_api/cli_types/installer.py I vaguely remember thinking that this module should be split up. Maybe.

It's late and I'm not up to going over the rest of these. I ended up getting a little distracted by how weird some of the code I've written looks. Like, normally, suite of a for loop makes up at least half of the lines of code in the loop.

I'm too tired to process a lot of this for now, so I'm going to just push this onto tomorrow.

Good night.

Weekly Roundup 2022-11-08

2022-11-08T05:00:00-05:00

Wednesday: My attempts to get back to working on MOTR were hindered by the fac that the tests ended up failing, but only sometimes.
Thursday: The bug that was breaking the tests got fixed promptly. I started renaming things.
Friday: I kept up with that, but tried to keep things short because I was writing that entry very late.
Saturday: I took a day off from this coding stuff, and I maybe should have written the post ahead of time, but I didn't.
Sunday: I ended up doing most of the work after I posted, which... not ideal, for various reasons, but it is what it is.
Monday: I did some final cleanup that was needed around the renames.

Next week, I've got a bunch of stuff happening during the week, so we'll have to see how I work around that.

Coding 2022-11-07

2022-11-07T05:00:00-05:00

After the last entry, I zoned out and finished the renaming. Towards the end, I realized that I'd made some changes on the documentation topic that touch code not yet released in the coverage topic. So, to do The Right Thing in terms of the new interfaces, I switched to the coverage topic and made updates to just those classes, then merged and fixed the conflicts. The merge was pretty painless, and I'd really like to attribute it to using Mercurial, but realistically, it was probably because I made sure to get it out of the way as soon as I could.

Anyway, next up I'll either be carefully going through each individual file, or I'll think of something new to change that's more targeted.

Either way, I'm done for now.

Good night.

Diary 2022-11-06

2022-11-06T04:00:00-05:00

All right. I spent most of today just kind of relaxing, but I'm feeling okay right now. Sadly, there isn't time to do much.

I'm going to try to make some updates to MOTR after I publish this, but I'm not feeling up to writing much coherent yet.

I'm just going to cut this here, and encourage everyone not to push themselves too hard, and then I'll try to follow that advice myself.

Good night.

Diary 2022-11-05

2022-11-05T04:00:00-04:00

I was only so-so at taking things easy for most of the day, but oh boy am I taking things easy right now.

I ended up feeling up to doing some work on the latest MOTR rewrite, but I'm going to take a break from that now.

If I can focus tomorrow, I'll get to work on it again. If I can't, oh well.

Good night.

Coding 2022-11-04

2022-11-04T04:00:00-04:00

Time management messed up again. In order to recover, I'm going to need to not work on MOTR tomorrow, then work on it Saturday afternoon.

(Regular reminder that the dates on these posts don't make it very clear which day of the week I'm writing them on.)

Anyway, I am continuing to rework the names in MOTR's core. I shouldn't try to go into too much more detail now. (In fact, I should stop working on it for now, but, eh.) I'll try to summarize how things are going in the entry that I write on Saturday, which will be dated Sunday.

Good night.

Coding 2022-11-03

2022-11-03T04:00:00-04:00

I figured out what was breaking MOTR, and filed a bug upstream. I'm not proud of the bug, but it got the job done, many thanks to the maintainers.

So, I've got a few minutes, let's see about breaking the tests on purpose. The big overall change I want to put in for legibility, is to switch from adding "requirements" to a "registry", whatever that is, to putting "facts" into a "compendium".

...

I've gotten the code to typecheck, but there is much more to do, and no time now to do it in. Over the next few days, I'm going to divide my time between getting other checks to pass, and doing more renames. There is plenty more to do there, and it feels good to have finally gotten started on all of this.

Good night.

Coding 2022-11-02

2022-11-02T04:00:00-04:00

I thought I was going to be able to get a little quick work on MOTR, but it ended up not being the quick work I expected.

It seems Pytest made some changes that broke a plugin I was using, until I added an additional requirement, but now I'm getting test failures that feel totally random, even though they're sort of reproducible.

Okay, no, somehow these tests that execute a linear series of actions are behaving non-deterministically. This is going to bother me for days.

I don't deserve this treatment.

Anyway, I'll work on this later, when I'm better-rested to handle these mysteries.

Good night.

Weekly Roundup 2022-11-01

2022-11-01T04:00:00-04:00

Wednesday: I tried to implement a little of what I want for the parser code.
Thursday: Bad time management.
Friday: I tried to rough out a bit more of the support code for parsing.
Saturday: I watched MST3K.
Sunday: I... worked on tabletop roleplaying stuff, apparently.
Monday: I didn't do much worth writing about.

Next week, I want to get back to messing around with MOTR, now that my laptop seems to be working somewhat reliably.

Diary 2022-10-31

2022-10-31T04:00:00-04:00

I ended up messing with other stuff today, and I think I made some interesting progress, but I'll have to give it a bit more time.

I'd like it if I could get myself to put in work on the stuff I have blog tags for, but at least I put in some work on something.

If I can get motivation to work on code stuff, I should try to make changes to MOTR, since I came up with some changes that should clarify things, or at least make them less generic.

Anyway, I let things slip badly again, oh well.

Good night.

Diary 2022-10-30

2022-10-30T04:00:00-04:00

I ended up not doing much today, so I'll just pretty much accept that.

One thing I did work some on is something that I've started working on recently with my wife. I'm trying to start a game of Ironsworn: Starforged, with the idea that I'll be playing it mostly solo, but asking my wife for oracle decisions. Right now, I'm pinning down the setting and my character's backstory. This morning, I copied in the setting truths from my notebook to my laptop, so now I can work on collating them and figuring out the implications.

Having my wife make the choices worked out all right in setting up the truths, but I haven't yet tried it with much larger tables.

Anyway, I'm reading over the beginning of the rulebook again to see if there are elements I should be incorporating into my notes. A bit more work on this stuff, and I should be able to get started.

Anyway, that's all I've got, and I really shouldn't try to come up with more.

Good night.

Diary 2022-10-29

2022-10-29T04:00:00-04:00

More MST3k tonight. That movie was a lot (in terms of the special effects sequences) and a little (in terms of the plot).

I'm actually not quite sure when I'll be able to properly ramp up in the things I want to work on. Stuff is happening this weekend, I think, and over the next few weeks. Maybe I should just work on getting stuff done in the morning.

Either way, nothing's happening now, and if I don't wrap up, nothing's happening later.

Good night.

Coding 2022-10-28

2022-10-28T04:00:00-04:00

Minor technical issue: worked around!

Time management: still in terrible shape!

The next step in my quest to carefully design the environment on which to build stuff in Lua, is to figure out how I want to do stuff like "annotate fields with their name".

My general feeling is that things should work like { { field = {...} } }, where there could be many "field" tables, and the {...} corresponds to a table that gets passed to converter_function_passed_by_client_code(table.unpack(v)). That way, the current behavior of the enum module could be approximated by mapping function(field) for k, v in field do table.insert(v, k) end return field end over ipairs(fields_table).

There are some issues here. I think I want to have similar power to table.insert, but doing it like this means that the field tables get modified in place, then collected into a new table, which feels not-quite right. Like, either it should return a new table without touching the original, or it should update the original table and return the same table. I don't know which of those I prefer from a pragmatic perspective. (My gut says "avoid mutations as much as possible", but if I felt that was worth basing my pragmatic stances off of, I wouldn't be doing this in Lua.)

The "don't look inside the sausage factory answer" is to come up with an alternative to mapping over an ipairs that just loops over the table and assigns the result of the mapping function over the existing value, regardless of whether they're the same.

I'm cutting this entry off for now, but now I've got the next question for later:

Is that idea something I'd want to reuse in other contexts, or just some form of utility function I'd want for building up this kind of data?

I'm going to need to phrase this question very carefully, due to the nested structure of the tables, and the number of functions I'd like to have in play. I'll take about a day to think about this, at least. For now, I need to get ready for bed.

Good night.

Diary 2022-10-27

2022-10-27T04:00:00-04:00

My time management slipped again. There's no way for me to get out an entry that's any better than this. I still want to post something daily, technology permitting, but sometimes it's not going to be all that much.

Good night.

Coding 2022-10-26

2022-10-26T04:00:00-04:00

Okay, my laptop is still giving me some technical issues, but I'm going to put them in the "annoying, with a planned workaround" bucket for now.

Anyway, I'm still not ramping up super quickly, so for now, I'm just working on parts of the stuff I was planning out two days ago. I decided to start small, and implement the "ordered fields" helper. With that, I can have my arbitrary criterion satisfied ("I shouldn't need to quote the field names when I define them") without having to explode out the creation process into a bunch of metamethods that may or may not fully make sense. While I was at it, I made sure it's possible to have arguments that don't do anything in terms of the final result, so they can act as spacers between groups.

I'm going to have to wait to see what I think of using this code, because it's late, but, like, just make a bit of progress at a time.

Good night.

Weekly Roundup 2022-10-25

2022-10-25T04:00:00-04:00

Wednesday: [FOOTAGE MISSING]
Thursday: [FOOTAGE MISSING]
Friday: [FOOTAGE MISSING]
Saturday: I had everything together enough to be comfortable trying to post again.
Sunday: I tried to get back into the Crafting Interpreters stuff, and promptly ended up kind of confused.
Monday: I decided to try biting the bullet and settling on an enum representation that wasn't absolutely as theoretically concise as possible. Note that I just decided, I didn't actually write any code. I'm not coming back into this stuff at full speed.

Next week, I'm going to try to ramp back up, but maybe I'll just mess around and see if that makes me feel better.

Coding 2022-10-24

2022-10-24T04:00:00-04:00

I know it's pretty basic, but every time I think about this enum stuff, I end up going in circles.

I'm currently leaning towards "the stuff I'm thinking of doing with metamethods is too confusing at every level", so I'm considering other ways to have names in order. Like, let's compare...

local ordered = fields().abc.def.ghi  -- Complicated under the covers

local ordered = fields{{abc = {}}, {def = {}}, {ghi = {}}} -- A bit wordy

On reflection, it is wordier, for sure, but it's mostly going to end up on distinct lines anyway, and it gives me a convenient way to make things that contain their name: define a function to process each table, and run it over the table.

I think that gives me something good to try next time around.

I wish I'd gotten further, but to be honest, I'm still not totally sure my laptop isn't going to suffer some horrible failure, only mostly sure.

Anyway, I need to get ready for bed.

Good night.

Coding 2022-10-23

2022-10-23T04:00:00-04:00

I'm not going to get too far with anything tonight, but I want to refresh my memory on the Earley parser stuff. Looking at the last entry in this tag through the website, I see that I was in the middle of writing some changes to the enum modules I wrote.

My notes there just talk about "taking an iterator", so that's just, like, okay, take the values returned by an iterator function, and then feed them into the... ah, good, the previous Lua post actually explains this stuff.

Let's see, what do I want to call this stuff?

The table that either accumulates or chains names and potentially values can be called fields. From a purity craze perspective, I'd try for the chaining, but it's probably easier to implement with it just accumulating into a table. Remembering that the fields need to be required to be strings.
From there, the opairs function I've already written can be used to extract the data.
Something in there needs to handle defaulting arguments that aren't given, but I'm not sure what's the best way to handle that.

I think I need to do some planning on paper to properly hash a lot of this out.

Anyway, it's late, and I need to wrap things up. It wouldn't do to stay up too late on this laptop just as soon as it becomes possible again.

Good night.

Diary 2022-10-22

2022-10-22T04:00:00-04:00

Well, that sucked. Several days after my laptop appeared to be working again, it started showing the same issue as before. At that point, I gave up on it.

I'm writing this entry on a new laptop that I installed linux on from scratch. Here are some notes:

Not going to get anywhere without disabling secure boot in the BIOS.
I had a number of issues around resume, that may or may not be fixed. The last thing I tried was to switch from hybrid to discrete graphics in the BIOS, and that seems to have fixed horrible screen issues on resume. I also disabled the nvidia suspend and resume services, which may have been causing a different set of resume issues. The other thing I kept was to switch to the proprietary drivers.

No obvious problems yet; hopefully I'll manage to publish this entry tonight.

Anyway, the broad takeaway from this isn't any particular technical advice, but "Try to avoid having to do non-mainstream tech stuff to your main electronic device when you don't have some kind of fallback available."

Good night.

Weekly Roundup 2022-10-18

2022-10-18T04:00:00-04:00

Wednesday: I tried to figure out a proper middle ground between Python and Koka for representing variables.
Thursday: I thought a bit more about that, and concluded I need some more experience with other programming languages.
Friday: I had a bad day for writing/working, so I just kind of punted.
Saturday: I punted deliberately.
Sunday: My laptop got broken so badly that I elected to reinstall the OS from scratch. Getting this done and recovering most of my files took most of the day.
Monday: I wasn't quite done with restoring everything.

Next week, I'm going to try to ramp back up on NABTO related stuff and MOTR. I'll be keeping an eye out for any performance degradation like my laptop was experiencing like before it really broke, but that doesn't seem to be happening. Which, on the one hand, is good, but on the other hand, I don't know which factors to worry about, since it's mostly the same files, on all the same hardware, and all that's changed is that the system is freshly installed with a new distro version, and the disk layout is different.

Diary 2022-10-17

2022-10-17T04:00:00-04:00

I'm still reconstituting my laptop, so I'm not pushing myself too hard yet. I've taken care of most of the low-hanging fruit, and the backup did a lot of work, so there are only a few notable omissions that I'm just going to leave be for now.

I need to cool off properly, so I'm going to take that time out of writing this entry. Hopefully, I'll be able to ramp back up soonish.

Good night.

Diary 2022-10-16

2022-10-16T04:00:00-04:00

I spent literally all day wrangling my laptop back into a usable state, so I'm not going to do much more for now. (I haven't actually confirmed that it'll stand up to my typical usage, but it's holding up to stuff that seems comparable to what I was doing when everything broke down.)

I did make some progress planning renames for MOTR, so I can get on with that tomorrow if I'm actually done with fixing things.

(I am not done with fixing things. Publishing this entry is going to require a bizarre command-line invocation, and possibly changes to the publishing script to make it more robust.)

Anyway, I'm going to try to cool off now.

Good night.

Diary 2022-10-15

2022-10-15T04:00:00-04:00

I was planning to take things easy today, and that comes into play now. We're going to watch a bad movie a few minutes from now, and I'm just going to relax.

Tomorrow, I'll try to have things together, but the plan for now is to chill out. I'm not going to overthink other stuff to put in this entry.

Good night.

Diary 2022-10-14

2022-10-14T04:00:00-04:00

Ugh, where does the time go? Both in the day, and between the months?

Anyway, I shook things up a very small amount today, and that may or may not explain why I messed up my end-of-the-day schedule. An alternative explanation is just that I don't have anything new to say about NABTO currently, and so I ended up kind of stalling for a while. Perhaps there are multiple factors in play.

Anyway I didn't do stuff worth writing about today, but I can plan things out for the next few days:

I'm going to take things easy tomorrow.
After that, I'm going to either catch myself back up on the Earley parser, or start doing programming language tutorials. Ideally both, I think.

Anyway, I should not be writing this this late.

Good night.

Coding 2022-10-13

2022-10-13T04:00:00-04:00

After writing some more jank Koka, I believe that the state of local variables is the same after each call to resume() in Koka. Maybe there's an obvious reason that it has to be that way, but I was frankly pulling my mind in too many directions at once to be able to tell.

So, let's think some more about how variables should act. The big hurdle if I do go with "all stack variables are predictably-sized fat pointers to arbitrarily shaped objects on the heap, and there's a reference count somewhere", is that it gets kind of hard to to write a pure function, at least I think so.

Supposing we have a list/vector type that works kind of like in Python, because I think I want that. It's possible to express read-only semantics for the fat pointers here, but it doesn't actually help, because something else can take a reference to the object and mutate it. This is what has me thinking "hm, maybe capabilities would be useful somehow". Like, if a function relies on val, box, and tag type references to external data, then it shouldn't be possible to break purity by changing the objects behind those references, because they can't change from the perspective of the function.

This is complicated, so I'm going to try to break it down into reasonable steps.

"Hey, I want a language that's similarly dynamic to Python, but it has an effet system."
"Well, it's not obvious how to reconcile the effect system with how Python handles writing functions, so it doesn't make sense to work on the object system before addressing that."
"So, I want to be able to write a Koka-pure function that relies on module-level variables."
- This has some weird implications even before I get into trying to implement it, because it implies some way of saying "unlike other functions, this functions is not callable immediately after it is defined, but only after some external condition is met".
"I bet something like Rust or Pony could help with this."

So, if I want things to work the way I think I want them to, then I need to be able to implement the following setup:

Variables that are referenced via (probably fat) pointers, to data on the heap that contains some necessary metadata, and the actual data.
The ability to construct mutable variables.
The ability to define functions.
A basic type system.
A basic effect system.
- Which handles mutability of shared state.

I won't say that's a lot of the work, but if I can't do that much, then I need to reconsider the entire big picture of the design.

The near-term stuff to do goes as follows, but not in any particular order:

Take more of these notes
Go through Rust tutorials
Go through Pony tutorials
Get back to Crafting Interpreters, and therefore, my Earley parser yak shave.

It's good that I have goals, but next up, I need a plan to make sure I actually work on them. I'll see how that goes. Anyway, no more to do for today.

Good night.

Coding 2022-10-12

2022-10-12T04:00:00-04:00

While I let renaming stuff for MOTR percolate a little, I'm going to consider some thoughts about how NABTO "should" work. The current topic: how even do variables work?

Let me see if I can explain what has me so unsure. Consider the following Python code:

def getter_and_setter(value: int = 0) -> tuple[
    Callable[[], int], Callable[[int], None]
]:
    def getter() -> int:
        return value

    def setter(new_value: int) -> None:
        nonlocal value
        value = new_value

    return getter, setter

Now, I understand Koka just well enough to expect that the relevant type signatures in an equivalent function definition would look somewhat different. Let's see how things shake out...

So. This doesn't compile. But the error it produces is very informative.

fun getter-setter( initial_value : int = 0 ) : _ (() -> _ int, int -> _ ())
  var value := initial_value
  ({ value }, fn( new_value : int ) { value := new_value })

It would be worth figuring out if it's possible to get this to compile, because I think I want code like this to work in NABTO, but I also want it to have sensible effect typing.

Anyway, the basic thing I learned about Koka from writing that code is that Koka distinguishes between local variables, and heap variables, in terms of the effect system. My initial, probably faulty read of the compiler error is that a variable has to be allocated on the heap in order to close over it. I'd be more confident of this, or alternatively know that I have it wrong, if I had a better understanding of how variable allocation is controlled in Koka.

Anyway, this may or may not be moot, because one thing I'm not sure of is whether values in NABTO should act more like Python values, or Koka values. I'm leaning towards Python (heap allocated, reference counted), but I'm not sure what knock-on effects that has on the design.

In any case, my thinking on the above code snippets is that equivalent code should be possible in NABTO, but it should have effects attached to the types. This is because the returned functions cannot be pure in the Koka sense, so they must have some effect other than raising an exception or diverging.

My intuition about this is, I need to get some experience with how variables work in Koka, and, say, Rust and Pony. I believe that would cover a good portion of the relevant design space.

Anyway, I need to wrap up for now.

Good night.

Weekly Roundup 2022-10-11

2022-10-11T04:00:00-04:00

Wednesday: I took care of some confusing names in MOTR's core.
Thursday: I updated a type definition and set myself up to fix code for quite a while.
Friday: I got most of the way (or possibly just "most of the way", I don't quite remember) fixing the type errors.
Saturday: I fixed up the types, and set myself the next level of work: fixing the tests.
Sunday: I finished a few other sundry checks and got to work prioritizing other tasks.
Monday: I drilled down from that into prioritizing the TODO comments littering the code.

Next week, I'm going to try to nail down the details on some of these TODOs. And whatever else I feel like. Maybe try to sketch some stuff out for NABTO.

Coding 2022-10-10

2022-10-10T04:00:00-04:00

The first thing on that list I put together yesterday was fixing # TODO: comments. Instead of jumping in at random, I'm going to try to prioritize them. Some are good to work on now, and others should go on the "release" topic once I've fixed up the coverage.

Soon:

Reconsider the name of "requirements", at both a type and module level.
Reconsider the name of "registry", at both a type and module level.
Rename (Input|Output).path to value.
Reconsider the names of Input and Output.
I was thinking of renaming Target and related classes to Outcome, but maybe what I want to do is call them Artifact, or something? I feel like any choice I make is going to have some terminological issues. Like, Target isn't inaccurate, but doesn't communicate much. Contrastingly, Artifact gives a clearer picture of the general usage, but doesn't really fit some critical cases. Looking at both of those choices, Outcome actually just feels like a bad compromise... Basically, this name should refer to something that can be thought of in three related contexts:
- An effect of a task that can be reasoned about in other contexts, an Outcome.
- A condition that must hold in order for a task to start, a Prerequisite.
- An externally visible result that the user can request MOTR to provide, currently a NamedOutcome.

Later:

Clean up the pip installer code
Reconsider the name of the base_cmd helper function.
I've got a TODO in the parametric module that I don't remember the context for what I was saying... Basically, one of the validation helpers on the parametric metadata is meant to determine whether the metadata can be used in an InputAccumulator. The case in question triggers when the multivalue_labels field has a non-None value. When that field has a non-None value, that means it contains a collection of labels. No labels except for the ones in that collection are allowed to take more than one. That means it's acceptable for the InputAccumulator to accumulate more labels, as long as ultimately, only the multivalue_labels take on multiple values. I think that means I can just remove the branch that emits the message? Maybe?
Speaking of InputAccumulator... Oh yeah, I added a bunch of classes and protocols, and now I don't know if my terminology makes any sense. But anyway, the only TODO comment I have in there is, I think, about how I'm calling a validation helper where the name of the helper doesn't match the context it's called in, which implies that I need to reconsider the name of the not_flex_out_because() helper, or write an explanatory comment justifying its usage.
There's a pretty basic and more-or-less self-explanatory TODO in parametric_command that's waiting for proper test coverage.
Bonus thought: cli_types doesn't seem like a good name?

I can make some of the "soon" changes easily enough, but others I feel like I need to take more time to consider them.

...

Update from the end of the night: I did the path to value renames because they were trivial, but I need to give some more thought to the others.

Good night.

Coding 2022-10-09

2022-10-09T04:00:00-04:00

I got MOTR to run on the new branch as well as it's going to, after fixing up a lot of code and tests. And now I'm figuring, why not just hit the rest of the TODOs next, and clean everything up as well as I can?

This is a little unfortunate. I did a bunch of work earlier today, but now I'm writing it up at the last moment, so I'm just kind of like "Yeah, I did a lot of stuff, a few hours ago, so it's not fresh in my mind any more."

I do remember that I did run into some issues with how I was doing mocking, but it was fortunately pretty isolated, and I came up with a not-great solution that's good enough. The fundamental issue there is that I didn't have "code that I owned" at the place that I wanted to mock. Actually, that does suggest a possibility for addressing this in a more principled way: create an internal-ish helper function that wraps the third-party code in question, and mock that. Before I get into that, I should see if there's a better way to write the tests in question.

See, one thing that I ended up discovering while I was fixing test code, is that I've let the test code kind of lag behind how I want to use this stuff, and when I make the sensible updates to make it consistent, it also gets a bit more robust to the changes I was making, which is fortunate.

So, here's the general idea for what kind of changes I want to make next:

Address # TODO:s.
Re-evaluate tests.
Maybe tweak the code a bit more.
Address all pylint messages. (Possibly by adding more ignores.)
Set up documentation generation.
Develop plan for extending test coverage on "release" topic.
Execute plan, then merge to "future" and address any conflicts.
Cut patch release.
Update motrfile from release.
Merge to "future".
Cut minor release.

That seems like a lot, but all of these quality improvements should hopefully make it easier to handle writing new tests against the "release" topic's code. Not in any kind of direct material sense, just that I won't get distracted going "But the code could be so much better", because it already is.

Anyway, I let things go too late, so I'm going to cut things off awkwardly and abruptly.

Good night.

Coding 2022-10-08

2022-10-08T04:00:00-04:00

Okay, the code in MOTR passes type checking. Now, I need to fix a whole bunch of test failures as a result of messing with all of the interfaces.

It turns out that fixing the type errors wasn't too bad, because I punted on "how do I tell the user what went wrong in the event of a task aborting the execution?". I think the answer there is to take some of the code I wrote for the custom build task, and turn that into proper library code.

Anyway, I had too much screen time today, and it's late, so I'm going to wrap up earlier than usual and try to recover.

Good night.

Coding 2022-10-07

2022-10-07T04:00:00-04:00

Another day of spotty time management. I did, however, fix all of the easy type errors around switching MOTR to using multiplexers. What remains now is the entry point that actually uses all of this. I'm going to need to make some decisions about how the data gets back out of the tasks, now that I've completely changed how it's handled inside them. This might need some new helper methods, we'll see.

Anyway, progress, that's good.

Good night.

Coding 2022-10-06

2022-10-06T04:00:00-04:00

I messed up time management, so let's see if there's anything I can do quickly.

Well, I can update the type definitions of a Task and break everything...

And now everything is broken, but it looks like generally a pretty simple fix. One thing I've held off on is deciding whether or not the Execution.multiplexer field should have a default. I can't think of a reason not to, but I want to hold off from committing to it for now.

So, plan for tomorrow:

Fix the messages out of Mypy. This will be interesting in terms of fixing the callers into the code, because I've just completely changed the interface for retrieving completed streams.
Decide whether I want to default the multiplexer field.

Once those are taken care of, it's time to start documenting until I find something that offends my sensibilities again.

Anyway, it's way too late and I'm going to try to get ready for bed as soon as I can.

Good night.

Coding 2022-10-05

2022-10-05T04:00:00-04:00

All right, let's see... I completed a round of renaming, but I still want to work through more.

Right now, I'm trying to figure out what to name two internal and low-level classes, because the current names are really unintuitive.

Basically, one class represents, I guess, the ambient state of a single run of MOTR, relating the registry, a reporting interface, the results, and a reference to each running task. Later, it'll have a stream multiplexer attached to it.

That class uses another class as a helper. The point of the helper class is to wrap a bunch of helpful boilerplate around the execution of a task; this makes it easier to write task classes or functions, because there's no reason to change this behavior, so third-party (ish) code shouldn't be responsible for implementing it.

I ended up calling them Execution and Step, and renaming some internal classes. The results are leaps and bounds beyond what there was before, which were the same names as related, but completely different classes elsewhere in the application. (Some of those classes were also renamed, but just making the name unique wasn't enough.)

That's some good work for today, but there are many more passes to do. Before I move around, I should probably work on beefing up the documentation here to make sure I still like the names after I've tried to really use them.

Anyway, it's late and I want to wrap up for tonight, so...

Bye.

Good night.

Weekly Roundup 2022-10-04

2022-10-04T04:00:00-04:00

Wednesday: Now that I can make any change to MOTR that I want to, without worrying about breaking the motrfile, I fixed what is basically a tiny cosmetic issue with the code.
Thursday: I started laying out the types that I want to use with MOTR going forward.
Friday: I realized that I wasn't done laying that stuff out.
Saturday: I fixed the most obvious issues I saw with my initial ideas there.
Sunday: I, unfortunately, did things all at the wrong time, but I put together a prototype of those new classes.
Monday: I took the time to figure out what was definitely wrong with that code that I prototyped as fast as I could, late at night. Then I started renaming classes and attributes in the broader codebase. Everything in this topic is all messed up, but that's what it's for.

Next week, I'm going to continue on with the cleanup, and hopefully try to handle screentime and sleep better.

Coding 2022-10-03

2022-10-03T04:00:00-04:00

MOTR status: I fixed the obvious issues with the code I prototyped last night, and then I got excited about doing renames, and everything is broken, but I'm already happy with how it'll be when I get it to stop being broken.

I'm not going to try to push this past the hour mark, so that's about it.

Good night.

Coding 2022-10-02

2022-10-02T04:00:00-04:00

Quick status update on MOTR, because my time management is backsliding.

I've drafted out the improved ideas I had for the multiplexed stream stuff. It all looks like it should mostly work, and my goal now is to make that statement less qualified. I need tests of the behavior, that cover all of the ways things could go wrong.

I think there are also some edge cases that I'm not properly handling, because I'm not sure what properly handling them looks like.

It probably wasn't the best decision to put in all of this work so late at night, but I wanted stuff to be put together and working now, dangit.

Anyway, no point in messing around any longer. I'll try to explain what I did in proper detail later.

Good night.

Coding 2022-10-01

2022-10-01T04:00:00-04:00

Okay, after I published yesterday's entry, I spent some time thinking, and found a way to make the public interface nicer, and the internal interface easier to implement.

So, the other side of things is how to set up the ReceiveStream interface. I was testing earlier today, and attrs classes, even frozen ones, appear to work fine as keys of WeakKeyDictionarys So, that'll work for my purposes, in concert with a few other options.

I don't have time to try this stuff out right now, since there are some other things planned for tonight, but I should be able to put it all together by tomorrow afternoon.

For now, I'm going to take things easy, as planned.

Good night.

Coding 2022-09-30

2022-09-30T04:00:00-04:00

I tried to dive right into the stream multiplexer stuff, and I discovered that my initial sketch of a design has... problems. So far, I've identified two major things that need to be addressed.

One is that I'm trying to express the underlying keys in terms of tuples of strings, but the public interface in terms of strings. So does the protocol just, like, have methods with both signatures?

The other is that I hadn't thought too hard about how to implement "a trio.abc.ReceiveStream subclass where instances are backed by a concrete multiplexer, and each instance pulls from the same memoized bytes, keeps track of its place, and awaits more data if it reaches the end".

I've got some ideas here, but I don't think they're going to come together in the next few minutes, so I'm just going to sleep on this for a bit.

Good night.

Coding 2022-09-29

2022-09-29T04:00:00-04:00

All right, I had some fun yesterday, time to have a bit more.

Right now, the definition I want to mess with is RuntimeAction = typing.Callable[[], typing.Awaitable[tuple[_result.Result, typing.Mapping[str, str]]]]. I want it to look more like Task = typing.Callable[[Something], typing.Awaitable[_result.Result]], where Something is an interface that takes a stream name and a stream.

So, since this stuff is pretty tightly wedded to trio internals, then we have that the "stream" argument must be a trio.abc.ReceiveStream, and there are a few operations that I want to support using them. One is for calling code to enter a stream name, and block until that stream completes, then cache and return the result. The other is to have some kind of "wrapper" implementation of the interface, that has a "prefix" that it adds to calls of the underlying implementation. So, let's suppose that stream names are non-empty tuples of strings.

And...

I got distracted by other things. So, let's plan this for later:

New module defining the interface, and two implementations: Concrete and wrapper.
Interface handles associating the streams to a key, and caching their data. Consumers of the interface should not interact with the stream directly after they pass it in.
Call it StreamMultiplexer or something.
The module goes in core.

I think this all makes sense. At least, I think so now. We'll see about later.

Good night.

Coding 2022-09-28

2022-09-28T04:00:00-04:00

All right, making changes to the future branch of MOTR. I could work on the documentation, or I could address some bits in the code that just annoy me when I think about them...

diff --git a/src/motr/motr_app.py b/src/motr/motr_app.py
--- a/src/motr/motr_app.py
+++ b/src/motr/motr_app.py
@@ -15,7 +15,6 @@ if typing.TYPE_CHECKING:

 # configuration defaults
 CONFIG = init_defaults("motr")
-CONFIG["motr"]["foo"] = "bar"


 class MOTR(App):
diff --git a/src/motr/templates/command1.jinja2 b/src/motr/templates/command1.jinja2
deleted file mode 100644
--- a/src/motr/templates/command1.jinja2
+++ /dev/null
@@ -1,4 +0,0 @@
-
-Example Template (templates/command1.jinja2)
-
-Foo => {{ foo }}

Aw, yeah, that's the good stuff.

The next thing I'd really like to look into is fixing up the signal handling interactions, but that could take a while and it's late, so instead, I'll figure out what I want to focus on first. Probably the revamp of runtime actions/tasks/whatever. That should be fiddly, and break everything before making it better, and I want to be sure it actually works.

First I get that working, then I get it documented. Then I start documenting things one level up.

For now, I'm going to wind down for a while. Still working on getting a good schedule together...

Good night.

Weekly Roundup 2022-09-27

2022-09-27T04:00:00-04:00

Wednesday: I planned out a more general way of creating "enums" for the Lox interpreter.
Thursday: I did a little work on that plan.
Friday: I got back to MOTR, and investigated the coverage gaps.
Saturday: I tried to come up with a good way to address some coverage gaps.
Sunday: I tried to come up with a different way. When I tried actually plugging the gap, the result did not bring me joy, which kind of calls into question how helpful these "helpers" are.
Monday: I decided to change tack a bit, by documenting the code as I want it to be, and introducing changes to the release-like code that accord with insights I gain from documentation.

Next week, I'm going to slowly work on spinning up documentation stuff. And maybe get back to Crafting Interpreters.

Coding 2022-09-26

2022-09-26T04:00:00-04:00

I was thinking about how bad it feels that I'm trying to push through covering code that I no longer fully understand, but, if I try to make the code clearer, that means breaking internal compatibility post-release, until I re-update my motrfile.

So, I can't update my code if I don't understand it, but I can't make it understandable without updating it.

Right now, I'm going to see if I can make a way around that conflict.

First off, I'm considering full test coverage non-negotiable. I find horrible bugs in non-covered lines shamefully often, so I'm going to keep that check on.

Now, going forward, I want full Pylint runs.

But here's the thing I'm going to try changing: I'm going to develop in multiple topics. Here's what I'm thinking: I branch a "future" topic off of the current horribly bloated main development topic. In that topic, I focus on documentation and code updates. Whenever I properly understand some high-level code, I switch back to the topic I'm on now, which is called, um, input-map, and appears to contain... the majority of the commits in the repository... and is nearly a year old.

All right, so first I'm going to try to close out the related bugs, make a tracking issue for test coverage, and make a new topic based off that. I'll disable pylint in the test-coverage branch, then make the "future" branch, and re-enable pylint there. This new topic is for source code changes only. Increases to test coverage go in the test coverage topic, and then get merged into future. The desired end state is that both topics pass MOTR completely, and future is descended from every commit in test coverage. That way, I can cut an 0.1.6 release from test coverage, then create a new tracking issue to update the motrfile. Merge that into future, address the conflicts, and cut 0.2.0 from the future, turning it into the past!

It couldn't be simpler! ... To gloss over a large amount of work, especially for just one developer, but whatever, I feel better about this plan than about what I was doing before.

Let's break this down a little. Regardless of what I do in the medium term, it can only improve things to clean up the state of the issue tracker and get the towncrier files in order. I'm going to just list the bugs it looks like I need to close out:

51 through 55 are questionable, and I'd rather put them in the 0.2.0 push, in that I've started on them, but I'm not entirely done until the motrfile is rewritten. Note that this is excepting 53, which is complete, and the thing that the current topic is named after. I also haven't done 66, and that kind of needs to happen at some point.

56, 57, technically 58, technically 59, 60, 62, 63, 64, 65 I think, 77 is started but I'll be able to do much better when I bump the Python version requirement.

So, it'll be pretty quick to get that housekeeping done, so I'll take care of it now...

On further reflection, I ended up only marking a few issues as properly done, but I should be able to move forward now.

Okay, I made the topics, and... I am having trouble. I forget how to make changes show up in multiple topics, or if that even makes sense, so let's see...

I'm very confused, and this might not be an intended workflow. A bunch of things I tried didn't work, so let's see if I can reason my way to something that does.

...

Dangit. I was doing everything fine, but I was getting tripped up because the "future" branch is labeled with its issue number, and I already had a documentation issue, so I just used that, so the merge commit is supposed to be in the lower number topic. This will mess something up at some point, I guess.

But yeah, this all seems to work the way I wanted it to, when I'm not making it overly fancy and confusing myself.

I think I've been trying to pack too much work into my days, so I'm going to space things out.

All right, time to wrap up. Next time I get to work on this stuff, I'm going to try out some rewrites I've had in my back pocket for months, just waiting for a chance to mess with.

Anyway, I should start winding down now.

Good night.

Coding 2022-09-25

2022-09-25T04:00:00-04:00

With the benefit of a good night's sleep, I realize that it doesn't really make sense for MOTR to support flake8-json, so my next guess for "what is this helper function for" is a pytest plugin of some kind.

Ideally I wouldn't be do this kind of guesswork because I would have written down my reasoning somewhere I can find it, but the best I can do is to try to come up with a plan to avoid this scenario in the future.

Anyway, looks like there are a bunch of candidates in terms of Pytest plugins... Do I see anything I like...

pytest-codestyle isn't something I'd use, but it fits with how MOTR works better than other things I've looked at.

Hm...

The integration looks good in terms of what the code does, but using these interfaces is horribly awkward, which is on me, either then, or now.

It's a pain, but I don't think I can progress on this unless I get the documentation written so I can see a summary of how and why any of this stuff is supposed to be used.

I don't know how much I'll get down tomorrow, but I'll maybe try it over the next week.

For now, I need to wrap up.

Good night.

Coding 2022-09-24

2022-09-24T04:00:00-04:00

Let's see if there are any quick hits I can do with MOTR. For now, I'm going to sort by least lines missing, and skip the Mypy plugin, as well as validation logic. So, by those standards, the first module to hit is... python_helpers. Okay, let's see what the deal is...

I've got two helper things in here, and I can't remember precisely what either of them is for. One of them extends the command to be given, and the other creates a Static containing an option for some program.

With the context I'm seeing for the latter one, it looks like the point of that is to bundle up a flag and a plugin package. I'm blanking on which thing this could be for, but it kind of sounds like a flake8 plugin might fit the bill? Nothing really jumped out at me as "yes, that is what I was thinking of" when I checked, but flake8-json looks like a reasonable thing to support.

Sadly, time, and so I'm going to leave this for myself to work on in the morning

Good night.

Coding 2022-09-23

2022-09-23T04:00:00-04:00

Ugh, I didn't get things done with the Lox code today, so instead, how about I try to refresh my memory about MOTR's code on the way into the weekend?

Okay, I've got thirteen files that need to have improved coverage:

motr._api.cli.build: Missing some failure paths that it's going to take either a rewrite, or some "interesting" mocking to hit...
motr._api.cli_types.command: I think this was involved in all of the renames I wanted to do. There are several paths missing. Resolving Executable values as environment variables and extra IO, and some unimplemented paths that look obvious from context... There's also a helper function that needs to be either deleted or renamed.
motr._api.cli_types.command_builder: Missing error case, and extending an OptionBuilder, which seems like a sensible thing to do, I think...
motr._api.cli_types.flex: Missing unprefixed parent argument, protocol methods on some of the classes, FlexOut with no parent.
motr._api.cli_types.input_accumulator: Missing some validation code.
motr._api.cli_types.installer: Missing some validation failures.
motr._api.cli_types.not_output: These functions are validation helpers, and just need to exercise some failure and happy paths. Some of the names should be updated as well, which should have mercifully little fallout.
motr._api.cli_types.parametric: This is one of the major support modules for the new system. It's mostly missing coverage on its validation paths, but it's also got partial coverage on a block that I'll need to think about how to address...
motr._api.cli_types.parametric_command: Needs to use one of the helper classes it defines, and it's missing an error path.
motr._api.helpers.python_helpers: Missing coverage on a helper method and a helper function. Hopefully it's not too hard to determine when they'd merit usage.
motr._api.installers.pip: This module is pulling double duty and should be pulled apart, but in any case, it's mostly missing validation, and the usage of a Python version specified as a string.
motr.validators: A validation helper that needs to raise a validation failure, have its documentation retrieved when the decorated function is documented, and when it is not documented.

Outside of the main source tree, there's also some Mypy plugin code missing a branch that I'll need to know Mypy better to know if I should write a test or convert it to an assertion.

Aside from the plugin code, I could probably get okay results just writing tests for the files in ascending order of absolute coverage misses. I just need some order to work through this stuff, that makes enough sense that I don't second-guess myself.

Anyway, it's late, and I want to wrap up.

Good night.

Coding 2022-09-22

2022-09-22T04:00:00-04:00

I let time get away from me today and I need to shut things down asap to recover, so I'm just going to list what I did from the stuff I laid out yesterday:

Prototype of helper function to create a table that adds names in order, and can optionally associate values to those names. (Missing: special casing for numerical indices, and detection of duplicate indices.)

Still to be done:

New enum module that uses a single-shot callable table, taking an iterator.
enum class
Use the new enum module
Use enum classes

Good night.

Coding 2022-09-21

2022-09-21T04:00:00-04:00

All right, let's see what I can manage for the Earley parser setup. I'm focusing on how to represent the rules, which is basically the following structure:

symbols are either nonterminals or terminals.
terminals are token types, and this might finally be what convinces me to try to wire up enum and class stuff.
nonterminals are keys in the rules table, where the corresponding values are ordered tables of key arrays mapped to semantic actions.
semantic actions are just callbacks that take the corresponding matches and return something, which is considered to be the value of whatever matches.

Well then. If I think I need to combine enums and classes, let's see how that looks...

There are some other uses of enums that would be simplified if I could figure out some way to pass arguments to the constructor.

The conclusion that I ultimately come to is that my current enum implementation is placing too much effort on semantic minimalism in the calling context. It can express one style of things very concisely, but it's very difficult to deviate from this.

I'm going to try prototyping something that requires a little more syntax, but should work better with the general code base.

Hm, something's not clicking for me, and I think I need to work through some logic. There are basically several different ways to assign values to the members of an enum:

The value is a function of just the member's name
The value is a function of the member's position in the overall sequence of members
The value is a function of arbitrary data associated with the member

Now, if I want to keep up with my motif of "use iterators a bunch", I could say "instead of a table, the enum constructor should take an iterator, that produces indices, names, and optionally values; if the value is absent, it uses the name". Then, I can split the current syntactic sugar into a table that accumulates names into an array, a normal table, and applying iterator combinators over existing iterators.

Then, the question is, what combining this decomposed concept with the class code looks like.

So, the New Enum doesn't take an argument, it just returns a table that can be called, exactly once. This table, and the return value of calling it, is the enum. It needs to be smuggled into the class creation function, so it can be marked as the type of the instances that will be created when it is called, but it needs to be delegating to the original constructor.

Create the enum constructor
Allocate a local to hold the original class constructor
Write a function to store the constructor in the local, and return the enum constructor
Add this constructor to the class creation arguments, and then return the resulting class, which is actually the enum constructor (which is actually the enum)
The caller must now invoke the constructor with an iterator that provides triples of the form index, name, {args}

I guess it would kind of make sense for the enum constructor to set itself up like an ordered table, so that the opairs function I wrote can iterate over its members in order.

So, here are the things I need to write:

Helper function to create a table that adds names in order, and can optionally associate values to those names
New enum module that is based around a callable-once table that takes an iterator
Prove these out by replacing the existing enum usages; write any necessary helper functions
New enum class module that properly combines classes and the new enum
Update the usages to switch to the class-based enum

Sadly, I was a bit out of it today, so I've once again just got planning, and I don't feel like trying to actually care out these plans right now. We'll see how tomorrow goes.

Good night.

Weekly Roundup 2022-09-20

2022-09-20T04:00:00-04:00

Wednesday: I came up with some plans for object-oriented code in my version of the Lox interpreter.
Thursday: I implemented those plans.
Friday: I prepared to add some runtime correctness checks to the interpreter code.
Saturday: I added those checks.
Sunday: I decided that I was tired of implementing top-down parsers, for this, and in general, so let's see how far I can get trying to implement an Earley parser instead.
Monday: I started prototyping the support classes for implementing an Earley parser in pure Lua.

Next week, I'm going to try to keep up writing and publishing these posts earlier than before. It's not, like, a miracle cure, but I think it's helping me regain my focus.

Coding 2022-09-19

2022-09-19T04:00:00-04:00

I got some things done today. Basically, just a prototype for a set of functions for a data structure on top of Lua tables, to store keys and values, but also to maintain insertion order.

That should be part of what I need to be able to incrementally construct the grammar. I just need to write some more functions that delegate to that as keys in a higher-order table. And also to work out some way to represent sequences in a way that indexing is comprehensible. I've done stuff along those lines before, but I'm going to need to think about how I want to do it.

Like, I need a mapping from sequences of values to a canonical representation that (I think) needs to be able to retrieve the values.

Hopefully, I'll have a plan there in a few days. For now, I need to wind down.

Good night.

Coding 2022-09-18

2022-09-18T04:00:00-04:00

I was thinking pretty hard about trying to beef up my enum function, but initial efforts didn't work, and I decided that trying to combine enums and classes wasn't worth it for now.

Anyway, then I spent most of today in a haze, so I didn't get much else done before now. I have concluded that I want to change up the stuff that I try to do relative to the tutorial, not because I refuse to do things that way because I know better, but because I've done things that way, and I want a change of pace.

So, I'm trying to figure out what it would take to parse Lox with an Earley parser instead of recursive descent.

Now, I tried to get some hands-on experience with Earley parsers, um, at the beginning of this year, apparently... Based on this series, which I'll be working from again.

I'm going to take things as they go, so maybe I'll get stuff done tonight, but what I want most is to plan this stuff properly.

Let's see what the current Lox expression grammar looks like in Earley parser terms:

expression -> equality (identity)
equality -> equality ["!=" "=="] comparison (expr.binary)
equality -> comparison (identity)
comparison -> comparison [">" ">=" "<" "<="] term (expr.binary)
comparison -> term (identity)
term -> term ["+" "-"] factor (expr.binary)
term -> factor (identity)
factor -> factor ["/" "*"] unary (expr.binary)
factor -> unary (identity)
unary -> ["!" "-"] unary (expr.unary)
unary -> primary (identity)
primary -> NUMBER (identity)
primary -> STRING (identity)
primary -> ["true" "false" "nil"] (identity)
primary -> "(" expression ")" (expr.grouping or just identity, except it has to discard the parentheses)

Basically, each rule is going to be a non-terminal symbol on the left-hand-side, zero or more symbols on the right-hand-side, and a callback to construct the expression once it knows it has a parse.

(Now, if I can get all of this to work, that means I can drop the final "accumulate" step from the scanner and build everything in an iterator.)

Actually, to make this parse totally work, I do need to handle the EOF token somehow...

Anyway, I let this go too late, so I'm going to cut this here, and think about how to handle the data structures later.

Good night.

Coding 2022-09-17

2022-09-17T04:00:00-04:00

I've wired in a bunch of the stuff I was working on yesterday, and so now the interpreter code has a bunch of run-time correctness checks. Getting those checks in means that the code is relatively more verbose. It turns out I don't get to flex as hard on Java if I try to actually deliver all of the functionality that the book's code is relying on.

It occurs to me that, if I put in the work to rework things in a... somewhat confusing way, I can fit my "enum" code into the paradigm of my "class" code, and then I'd get access to isinstance checks for enums.

Anyway, I can't do that right now, because I've got too much else that I want to do right now. Hopefully I'll get it all together tomorrow. I'll at least try to plan things out after I post this. Once this is either implemented or scrapped, I'll try to get into the parser. I'm not ready to mess with that yet.

Anyway, best wrap things up over here.

Good night.

Coding 2022-09-16

2022-09-16T04:00:00-04:00

Reverse Polish Notation challenge is complete... ish... I didn't think of a way to disambiguate how many arguments an operator takes, so I just didn't implement the unary case.

I'm reading over stuff on parsing, and I'm not sure how exactly I want to handle it. It seems like I could set things up internally in a similar manner to the scanner, but I can't really give "function that produces a tree" an iterator interface in this context, so it might look kind of weird(er).

Anyway, I want to finish up soon, so I'm just going to try to tweak the interface for contracts, and add "decorator" support to classes if I get that done quickly.

...

Okay, contracts interface tweak went fine. Now for class decoration.

...

Okay, that code seems to be in place. My priority over the next few days is to shore up contract usage over the codebase. I may also want to add a destructured higher-order-function so I can skip some explicit destructurings in the protocol methods.

Actually, one last module to add, won't even test it yet, and then I'll post. It's a thing for generating read-only proxies around tables.

...

Okay, that's all set for now, but I need, like, a spell-checker for metatable fields :/

Enough messing around, time to publish and work on the other stuff I want to get done.

Good night.

Coding 2022-09-15

2022-09-15T04:00:00-04:00

All right, good work, me.

I've got a basic single-dispatch multi-function-y thing with no inheritance. Nice and lean.

...

Like, I know it's not fair to compare my Lua version to the Java version, since the Java code is constrained for pedagogical reasons, but it takes less Lua code to put together the functionality I need from scratch than it does to force Java to sort of have metaprogramming. Just look:

local ephemeron = require "ephemeron"

local class_from_instance = ephemeron()

local m = {}

local function make_destructor(slots)
  return function(instance)
    local instance_args = {}
    for i, v in ipairs(slots) do
      instance_args[i] = instance[v]
    end
    return table.unpack(instance_args)
  end
end

local destructors = ephemeron()

local protocol = function(implementations)
  local impls = ephemeron()
  for k, v in pairs(implementations) do
    impls[k] = v
  end
  return function(instance, ...)
    return impls[class_from_instance[instance]](instance, ...)
  end
end

local class = function(body)
  local slots = table.move(body, 1, #body, 1, {})

  local instance_meta = {}

  for k, v in pairs(body) do
    if type(k) == "string" then
      instance_meta[k] = v
    end
  end

  local function build(...)
    local instance = {}
    local args = {...}
    for i, v in ipairs(slots) do
      instance[v] = args[i]
    end
    class_from_instance[instance] = build
    return setmetatable(instance, instance_meta)
  end
  destructors[build] = make_destructor(slots)
  return build
end

m.class = class
m.protocol = protocol

function m.destructure(instance)
  return destructors[class_from_instance[instance]](instance)
end

return m

I'm not even using all of that code yet, and it's still shorter than the initial version of GenerateAst.java. And, of course, it's regular Lua. That said, there are a few things I want to add. I've got the weird local X = function construct because I'm planning to add contract enforcement to the public API, which will add a few lines.

This is all used like

local disinherit = require "disinherit"

return {
  binary = disinherit.class {"left", "operator", "right"},
  grouping = disinherit.class {"expression"},
  literal = disinherit.class {"value"},
  unary = disinherit.class {"operator", "right"},
}

and

local expr = require "lox/expr"

local printast

local function parenthesize(name, ...)
  local builder = {"(", name}

  for _, v in ipairs {...} do
    table.insert(builder, " ")
    table.insert(builder, printast(v))
  end
  table.insert(builder, ")")

  return table.concat(builder)
end

printast = require"disinherit".protocol {
  [expr.binary] = function(binary)
    return parenthesize(binary.operator.lexeme, binary.left, binary.right)
  end,
  [expr.grouping] = function(grouping)
    return parenthesize("group", grouping.expression)
  end,
  [expr.literal] = function(literal)
    return tostring(literal.value)
  end,
  [expr.unary] = function(unary)
    return parenthesize(unary.operator.lexeme, unary.right)
  end,
}

return printast

I'm going to have to think about this some, because setting up contracts as-is will do Bad Things to the protocol function. Like, if you put contracts on a constructor, as-is, that'll break protocols because now you don't have access to the base constructor. Either the protocol method needs to have some way of getting the "underlying function", or the class function needs to have some way of taking an optional callback to transform the constructor before it ever gets called. I'm inclined to do things the latter way, although, to get that ergonomic, it'd need some tweaks to the contracts interface, which is no big deal.

Anyway, it's getting late, although not as late as it's gotten before, and I want to wrap up, so I'm going to post this, consider this other stuff later, and try to knock out the Reverse Polish Notation challenge tomorrow.

Good night.

Coding 2022-09-14

2022-09-14T04:00:00-04:00

I'm trying something a little different today by writing this up earlier, so I have fewer excuses to stay up at night, because my sleep was just... so messed up. Anyway, I completed the multiline comment challenge, after a few hiccups.

So, next up is "Representing code", and it's actually a good thing I'm trying to write this earlier in the day, because I'm going to need to think about how I implement this stuff this time.

That's because this chapter talks about different programming paradigms and ways of laying out classes, and how to bridge gaps between different styles, and let's see how what I'm doing kind of messes with all of that...

I'm implementing as little as possible when it comes to "object-oriented features". Currently, the way I'm writing "classes" in my code doesn't handle any reuse. The point is more to avoid downstream consumers of an object needing to explicitly require a module to get at basic operations on that object. Another potential motivation that would get me to write something would be if I had polymorphism; that is, if multiple kinds of object implemented the same operation in different ways However, that's not what I'd use for what Robert Nystrom is using the visitor pattern for, for exactly the reason that he's using the visitor pattern.

The other issue here is that Lua doesn't do static type-checks (unless something really fancy came out when I wasn't looking), and missing fields just resolve to nil, so my best hope there is to error out as early as possible.

Thinking about these problems, a solution begins to take shape:

"classes" somehow involves a callable constructor, which creates "instances", and somehow provides access to the __index table for the type of the instances.
These constructors can use the contracts code I wrote earlier.
Those contracts can be defined inline using a function factory that takes a key name as input.
The "visitors" should be Python-style protocol functions like len. They should be defined unlike how Python does them. Instead, they should mostly use a table associating the constructor callable to the implementation for the instances of that callable.
Just realized the __index-like table should be an ephemeron table, which is no big deal.

The big thing I need to work out before I start writing code is, what kind of callable should the constructors be, and how it should relate to the __index table (which need not actually hook into the metatable system in that way).

Anyway, I'll ponder that for a while, and try to take things easy for the rest of the day.

Good night.

Weekly Roundup 2022-09-13

2022-09-13T04:00:00-04:00

Wednesday: I started working on Crafting Interpreters in Lua, again.
Thursday: Since I'm starting fresh, I realized I could vary up my approach a little.
Friday: I got some work done...
Saturday: I finished the chapter, but I decided I wanted to polish the code some before moving on.
Sunday: I made changes that make it possible to test the interpreter, which is convenient, since I'm using it as the entry point for all of the internals.
Monday: I got everything taken care of for now, except for the challenges.

Next week,

Coding 2022-09-12

2022-09-12T04:00:00-04:00

Okay, here's the deal:

Test coverage is maxed out.
Functions in the scanner module are rewritten to make more sense. (The problems with them were all my fault.)
I've gotten documentation sort of working, I think. I'm not sure what to document, but at least I've got the infrastructure to do so.

I'm not feeling super excited to document the core modules, since those are potentially going to get all changed around in later chapters. I guess I can look into documenting the supporting modules I'm writing.

For now, though, I want to take things easy and get back to this later.

Good night.

Coding 2022-09-11

2022-09-11T04:00:00-04:00

The code is a little tidier now. I created an object to act as a proxy to the file system and the shell, and converted the interpreter to delegate to that object. This means I should be able to test the interpreter without it trying to exit the test runner, which, maybe Lunit can stop those shenanigans, but I'd much rather just not bother. If I can cut this off before it needs the tooling to intervene, I might as well.

I fixed some errors that I introduced in the rewrite, so now the majority of this stuff basically works.

What's missing:

The remainder of the test coverage
- General test mocks for the host system abstraction
Documentation
- Figuring out what documentation should look like
- And whether it's compatible with my habit of returning functions instead of tables from modules.
Doing the challenges on a branch.

Anyway, that's that for now.

Good night.

Coding 2022-09-10

2022-09-10T04:00:00-04:00

The chapter on the scanner is done, but I'm not ready to move on. The test coverage is way too low, I hit all sorts of errors testing things manually, and there are a bunch of questionable choices that I made in the service of getting things done.

I ended up writing more code to interact with Lua's native protocols, and this just feels sensible to me. If you write code to work with an object system, then it works with that object system, but if you write code to transform iterators, then it works with everything.

I've been tweaking things pretty hard as far as auto-formatting goes, and I'm still not really satisfied. Currently, I'm using LuaFormatter with a bunch of custom configuration, which overall gives me most of what I want.

But yeah, I need to take a break from getting code in, and work on verifying and documenting the stuff that's in there now.

Good night.

Coding 2022-09-09

2022-09-09T04:00:00-04:00

I just put together what I believe is the minimal implementation for how I want the interpreter object to work.

I wasn't focusing on much stuff today, so I didn't have much time for this, and I'm dashing out this entry way too late.

Tomorrow, I'll try to finish up the chapter, and if I do, I'll write tests in order to get the coverage up, and maybe try documenting some stuff.

Anyway, I have to go, now.

Good night.

Coding 2022-09-08

2022-09-08T04:00:00-04:00

I was looking over my previous code for Crafting Interpreters, and I found an interesting comment.

-- Can this be switched to the iterator protocol?
-- Don't try yet. That kind of speculative refactoring can go... poorly.
function scanner_index:scan_tokens()

And I thought... It's a fresh codebase. Why not try it now?

:) (glitchy)

Dewit.

To do this right, I need to get a handle on just what this would entail. Basically, I want to think of jlox's scanTokens() method instead as a top-level function that takes a source string and returns a Lua iterator of tokens. Now, the end goal (for now) is to accumulate this iterator state into a table, so I put together a little module that I think looks reasonable:

return function(func, iterator_function, state, initial_value, _closing_value)
  local accumulator = {}
  local control_variable = initial_value
  while control_variable ~= nil do
    local loop_variables = {iterator_function(state, control_variable)}
    control_variable = loop_variables[1]
    if control_variable == nil then
      break
    end
    table.insert(accumulator, (func(table.unpack(loop_variables))))
  end
  return accumulator
end

I'm not sure if I'll actually use this, but it all basically makes sense. It essentially handles the Lua iterator protocol, and uses a function to condense the possibly-multiple return values into a single result.

Anyway, the basic picture of what I'm going for with scan_tokens() is to make that into an iterator that uses the source code as the invariant state, and packs the start, current, and line values into a table in the control variable. Then the methods translate into functions that take arguments of source and the control variable.

Okay, that all went smoothly, except for right at the end: The scanToken() method reaches out to the static error() method on the core Lox class, and that... It makes me feel... I don't like doing that...

Last time around, I passed the whole interpreter to the Scanner class I wrote, so it could call the method. And this is vaguely annoying to do with the way I wrote this, because I guess I'd need to change the state variable...

Okay, updating the state variable wasn't too hard, and now there's just two things I want to do currently:

Wrap the various bits where I construct a control variable in functions, so I know where to update if I add more fields to the control variable.
Make the Lox class that the state variable needs an instance of... exist.

But I can't work on those right now, because it is late.

Good night.

Coding 2022-09-07

2022-09-07T04:00:00-04:00

Well, I started grinding through the Scanning chapter of Crafting Interpreters, and as can be expected, there's just a bunch of "Okay, Java is doing this, so the rough equivalent in Lua is ..."

I'm referring a bunch to my previous attempt, because I don't think anything was fundamentally wrong with it, I just got a little... messy.

Oh, also, I got a wrapper written around one of the tools called lua-fmt, so now I can write complete garbage formatting and then tell my laptop to fix it for me. It doesn't mind.

Anyway, I'm coming up on a bit where I feel like I put a bunch of effort into it in the previous iteration, but I don't remember how that effort was directed. Like, what was I trying to do? Presumably, it had something to do with switching variables between being top-level locals and instance values? I can only assume?

In general, I'm not sure I trust these static variables. They feel... singleton-y?

Anyway, I'll try to get on with this more tomorrow. Hopefully writing some tests will illuminate the situation.

Good night.

Weekly Roundup 2022-09-06

2022-09-06T04:00:00-04:00

Wednesday: I'm not sure what happened before writing the post, but I did think about NABTO during it.
Thursday: I worked out some stuff that I'd like to work in Python's types, and, by extension, that I'd like to work in NABTO's types. It's likely to be pretty hard to implement, unfortunately.
Friday: I tried to translate the Lox interpreter to Rust, and gave up on that because that's just too many different not-familiar-enough things to juggle.
Saturday: I made an effort to learn tooling for Lua, and put together something that seems promising.
Sunday: I got back into Python code, and did a little work on MOTR, and way too much work on one of my other projects, trying to diagnose an issue that was ultimately either a bug in attrs or a limitation of Python; either way, not my fault, but not hard to fix once I understood what was going wrong.
Monday: After all of that, I was catastrophically tired, and didn't get much done. Oh well.

Next week, I'm going to try to get more done on various coding fronts. Per Lua's philosophy of "build the stuff you need yourself", I'm putting together some interesting scaffolding for development. I'll try to describe that some once I've really used it, rather than writing so-so tests for it.

Coding 2022-09-05

2022-09-05T04:00:00-04:00

I didn't actually get much done today. I did clean things up from yesterday a bit.

But, because I was up so late working on this stuff last night, I was totally wiped out today and couldn't focus on anything.

So, cleaned stuff up, and I don't want to repeat that experience, so I'm going to just take the L for today.

Good night.

Coding 2022-09-04

2022-09-04T04:00:00-04:00

Well, I tried to work on MOTR and another project today, and the results were so-so at best. Maybe if I had a pomodoro and switched off. Anyway, I did make some progress. I added a few lines of coverage to MOTR, and upgraded the Python version of the other project so I could use some newer typing features.

My plan with the other project is to strip out a shockingly large amount of code in the service of relying on the standard library for more things. Before I'm ready to delete that code, I need to get the replacement code to work, and as to how that's going... it's going...

...

Since then, I've gotten most tests to pass, deleted the majority of the rest, and the remainder are... confounding. It's some weird tension between, yes, the original intention of the tests is completely invalidated, and the documentation is all horribly out of date... But on the other hand, I'm staring daggers at the test code, and I can't see why the tests aren't passing.

...

I fixed it, and it sucked, and it was completely not my fault.

This is all true. Quick summary, since it's late?

Attrs tries to make get_type_hints work by populating the __globals__ attribute on the generated __init__. It does this using dict.update(), so further updates to the module namespace have no effect. Ergo, can't use forward references.

Fortunately, I was already calculating all of the necessary information for the old system, so I didn't actually need to figure out how to do anything differently, I just hooked up the seemingly-dead code, and the tests just started working.

That said, there's a bunch more to be done here.

Excellent. Now, SLEEP!

... Not sure I like your tone. Nevertheless...

Good night.

Coding 2022-09-03

2022-09-03T04:00:00-04:00

I got a bunch of tooling more-or-less working for developing with Lua. We'll see if it all actually works out, because I was just focusing on getting things to more-or-less pass. Right now, I'm trying out:

sphinx-lua with Furo, and using Invoke instead of the makefiles. I haven't actually tried to document code with this yet, and if it doesn't work, I'll just switch to LDoc. Getting sphinx-lua to run right took some weird additions to the requirements file, so I'm not super-attached to it yet, but given the ability to use Furo's dark mode, I'll try to take it.
lunitx and LuaCov. Notes on getting this working: I had to add LUA_INIT=require 'luacov' to the environment for the lunit.sh call, because I didn't see another way to make it load the module, but it all seemed to work, once I created a proper .luacov file. Update: while I was writing the stuff down below, I realized I could just call lua -llunitx tests/*, so I did that instead, because I'm not sure what the wrapper does in comparison, and I'd rather call Lua directly I think? Late-breaking update: That did weird stuff to coverage, so I switched it back. Not sure what was going on there... The other part of it was getting lunitx tests happy with how Luacheck wants me to write modules. Here's the pattern that I settled on:

local t = require "lunit"

local m = t.TEST_CASE "<test file name>"

function m.test_etc() end

return m

With this, I'm not creating any globals, and it's all working through lunitx's compatibility layers, so it can discover the tests. Now, I need to get to writing some code so I can test it.

Some of that code may end up being random bits of scaffolding, because, Lua. Anyway, I feel like I've written enough, and I don't want to delay this post any.

Good night.

Coding 2022-09-02

2022-09-02T04:00:00-04:00

Okay, so, I'm really not feeling great right now, so I'm going to explain what I tried to do, then I'm going to lie down or something for a while.

I mentioned Crafting Interpreters yesterday, and I decided to pick it back up. Now, I've got a few attempted projects that try to port the first half to other languages. One in Lua that's in a weird state because I left a bunch of the challenge code in, so that's not a great baseline. One in Python that I don't quite remember what I was doing, but I don't want to get back into Python code right now, because I'm on some kind of Quixotic quest for perfection with MOTR.

A quixotic quest for perfection, hm? That sounds like some of the descriptions of Rust. I thought, maybe I could pick Rust back up, and translate the Java code to Rust code.

Max could not.

I don't know whether it's that I'm, uh, rusty with Rust, that it's uniquely annoying to translate from Java to Rust, or just that I'm extremely tired, but currently I'm just not feeling it. At least I know this wasn't anything to do with the borrow checker, because I did not get the code far enough for that to matter.

In support of "I'm out of practice with Rust"; well, I just... am.
In support of "The translation is annoying"; so, I guess I should be converting these throws declarations to Results? Maybe? I'm not sure?
In support of "I'm extremely tired"; I am extremely tired.

There are a few different tacks I can and should take to deal with this:

Work through a proper Rust tutorial.
Read the current chapter to the end, developing a plan for translation.
Potentially, for now use a language I'm more familiar with, like Lua.
Get some dang sleep.

I know what I should be working on to start with, so I'm going to try to wrap up earlier than usual tonight.

Good night.

Coding 2022-09-01

2022-09-01T04:00:00-04:00

I'm a little out of it again, so I'm going to focus on one particular thing that I want in NABTO, but I also, just, want in general.

Okay, so this is seemingly simple to describe in plain language, but it looks to me like it inevitably brings in a lot of complication when I try to formalize it.

So, it's a type, that takes three variables. Two of them are normal type variables. The other one is... A higher-kinded type that takes two arguments. In Python runtime terms, it's easiest to express the whole type as a tuple, so let's go with that. Call the higher-kinded variable HKT, and then there are two possible ways to inhabit this type:

MagicType[HKT, T, V] = tuple[HKT[T, V]] | MagicType[HKT, T, U] + tuple[HKT[U, V]]

Where + between types indicates "the type of the concatenation of tuples of these two types" which I'm pretty sure isn't ever going to be a thing. Functionally, it's something like variadic generics, so it's as if we could define a union of variadic generics...

MagicVariadic[HKT, T, V] = (HKT[T, V],) | (*MagicVariadic[HKT, T, U], HKT[U, V])

Which is still not a thing that the syntax is likely to permit. (This would be used like tuple[*MagicVariadic[HKT, T, V]])

I could imagine trying to write a plugin that would make that all sort of work, possibly in conjunction with existing plugins. But I don't think that's happening until Python 3.11 comes out.

:) (glitchy)

Not that it's not fun to imagine people trying to process that crazy syntax, but you should probably explain what this is for.

All right, so, I've got two motivating examples, and one of them kind of reduces to the other, so I'll do the other one first.

Suppose HKT[T, U] is Callable[[T], U]. Now if you have a T and a tuple[*MagicVariadic[OneArgCall, T, V]], then you can loop or functools.reduce() the tuple over the variable to obtain a V.

(In NABTO, the effect type would need to be tracked somehow.)

Someone's going to want to know whether looping or functools.reduce() is better.

"Better". Anyway, I was messing around earlier, and using functools.reduce() was always slower than an explicit loop, but I was able to get them pretty close by binding functools.reduce() as a default argument. I'm not going to give actual percentages, because if the functions are heavy enough, they'll swamp the difference. So, if you find yourself in this situation, go with whatever solution you're more comfortable with, and be ready to profile the code if it does seem slow.

Anyway, the other place I ended up wanting something like this was in the graph code I was messing with. I had the idea of a composed edge, an edge that follows an Edge[T, U] to an Edge[U, V]. And why stop at one composition, hmm? Or allow the compositions to form some kind of binary tree. Better to just lean on the associative property and collapse them all together. I can't prove I need this from a performance perspective, but it's just unaesthic to allow arbitrary binary structures that all "mean the same thing".

(Actually, from that perspective, there definitely needs to be a canonical form.)

Let's chase that thought for a sec.

MagicTuple[HKT, T, V] = tuple[HKT[T, V]] | tuple[HKT[T, U], MagicTuple[HKT[U, V]]]

I just realized that the U is kind of coming out of nowhere in literally all of these. Throw "existential types" on the pile...

Hooray!

Suffering!

I know, I know.

Anyway, however it's represented, we then want the conceptual ability to map as follows:

Edge[T, U] -> Callable[[T], frozenset[U]] -> Callable[[frozenset[T]], frozenset[U]]

Just a bunch of Haskell-inspired nonsense that I'm pretty sure Haskell wouldn't even let you do.

There are two wolves inside of me. One of them knows, intellectually, that trying to implement all of this type stuff is probably a bad idea. The other one really wants it, come ooon.

Meh, at least it's only my problem if I try to do this.

Thinking over this, I think the way forward is to consider a bunch of problems separately:

Get back into Koka in order to better learn effect systems and to get a handle on the types.
Try to implement features of interest, such as my ideas about inheritance, in smaller, specific-purpose experimental languages.
Maybe start over with Crafting Interpreters or something similar.

Anyway, I'm tired and I have other things I want to do, so I'm going to cap this off for now.

Good night.

Diary 2022-08-31

2022-08-31T04:00:00-04:00

I have no memory of where the time went today, at least, not after I "fixed" the technical difficulties that kept me from publishing yesterday. Basically, my home folder was too big, so I made some drastic adjustments that fit with how I actually use the files that I moved around, so the result should be fine. We'll see. There's a bit more that I can do to alleviate the situation, but I don't need to. I probably should though.

Anyway, there's some work to be done on this still that's in other areas, and I don't want to think about that right now.

ANYWAY.

I've been thinking about NABTO, and there are some scary interdependencies there. Like, when you have imports as a runtime concept, then that means that the import system needs to handle st<h> effects, and I'm not totally sure what that looks like. It's possible that trying to combine "dynamic language" with "Koka-style effect types" will result in something completely unrecognizable for reusing code, but I'm not really sure yet. I don't want to give up on the idea before I've thoroughly explored it. Sadly, I don't have time right now, because it's late and I'm struggling not to make weird typos.

Anyway, that's enough time spent trying to write more of this post.

Good night.

Weekly Roundup 2022-08-30

2022-08-30T04:00:00-04:00

Wednesday: I pondered some facets of programming language design.
Thursday: I gave a terrible working title to the vague collection of language ideas I've been messing with in my head, and laid out some of the details of it, or at least possibilities of the details.
Friday: I tried to describe basic programs, and rather quickly ran into some esoteric-feeling issues.
Saturday: I rolled out the updated stylesheet and templates, at long last.
Sunday: I got work done on MOTR.
Monday: I did some more work on MOTR, and ran into weird snags writing tests that were technically (very technically, I feel) my fault.

Next week, I'm going to keep on with MOTR, and other stuff, and I've had enough technical difficulties before writing this entry that I frankly don't want to think too hard about it right now.

Coding 2022-08-29

2022-08-29T04:00:00-04:00

The package tests are improved, and I'm trying to get the Build tests working. It's slow going, because the pyproject-build command is failing. After I wired stuff up enough that I could actually see the errors, I'm pretty sure they're indicating that there's something wrong with the synthetic packages I'm having it build, but...

Wait. It's trying to find the package name instead of the module name.

...

Ugh, TOML, why?

Well, in any case, I shouldn't have too much trouble testing the error case.

You should explain.

Wh—

You should explain what was going wrong. After all, everything was spec-conformant except for your code.

Ugh, fine.

So, here's the code that was broken:

{
    # ...
    "tool.flit.module": {"name": module_name},
}

And here's what worked:

{
    # ...
    "tool": {"flit": {"module": {"name": module_name}}},
}

That dict is an argument to the tomli_w.dumps function. As I saw it, I had two basic options for how to generate the synthetic pyproject.toml file. Either I could substitute into a template, or I could generate the data corresponding to the contents of the file, and convert it to a string. I decided to manipulate the data, because I figured that would be less of a pain than setting up templating or a big format string. I'm... not sure that was the case, but at least now I've cleared what's hopefully the biggest hurdle to that.

See, the difference between those two snippets is, one works, and the other doesn't, and the one that works, works because it's writing data that corresponds to this block from the examples in Flit's documentation:

[tool.flit.module]
name = "nsist"

In TOML, this corresponds to a table containing a key called "name", and that table is described by one of the above snippets. See, TOML has a bunch of syntactic sugar, and those dots in the file don't correspond to the dots in the first snippet. They correspond to the table nesting in the second snippet.

So, before I fixed this, I was writing the table I needed to write, to some weird gibberish key in the top level, rather than the nice telescoping whatever that's going on there.

I'd read this in the documentation back when TOML was taking off, but I didn't think through the implications this had for generating TOML from data rather than from text.

So, there it is, one of the hurdles I had to clear was that I wasn't thinking hard enough about the format that my data was in. Happy?

Always.

O...kay.

Anyway, I let this go late and I need to get to bed now.

Good night.

Coding 2022-08-28

2022-08-28T04:00:00-04:00

Well, I got a little bit done on MOTR. I added some tests, and deleted the code that the test was exercising. I admit the test coverage for both versions of the code paths involved is... not great.

Tomorrow, I'll try to shore up those tests a little, then get on with the next major test coverage hole: running the Build action against synthetic data.

We'll see how that goes, but for now, I really really need to get to bed.

Good night.

Site Design 2022-08-27

2022-08-27T04:00:00-04:00

I gave the redesign a few days to sit, and made a few final tweaks. It's time to push it out the door! Time for a slightly wider central column, slight tweaks to the color scheme, more big chunky lines with rounded bits, and

And me!

Yep, I decided to spin off my whole unsettling smiley bit that I've done a few times since... a year ago, and turned it into one of those little character avatars that I see Rust blogs and other blogs use.

...

This was a good idea.

:) (glitchy)

This is a good idea!

In all seriousness, I think this has some potential for Missable Mysteries, if I ever, um, get back to it.

Anyway, I want to take care of some other things, so this is just going to be a quick entry showing off the changes.

Good night.

Coding 2022-08-26

2022-08-26T04:00:00-04:00

I've been thinking a bit more about NABTO, and I realized that there's something I need to address entirely before anything remotely object-oriented enters the picture.

Here is a Hello World program in Koka:

fun main()
  println("Hello world!") // println output

Here is a Hello World program in Python:

print("Hello world!")

The Koka code has the println() call inside the main function, while the Python version has it at the top level of the module. In Koka, module-level code is analyzed and acted on at compile time, while in Python, it's executed when the module is imported at run time. This means that either we need to restrict which effects are permissible at the module level, or we need to allow imports in NABTO to potentially emit arbitrary effects, once.

Hm, that "once" makes something occur to me. The import statement in Python and the require() function in Lua are, due to the way they memoize their results, kind of like delayed values that immediately get forced.

Okay, it just occurred to me that I probably don't want to go down this road. If the importing module is responsible in any way for handling the effects from the module that it imports, then that means that the values in a module can depend on where it's first imported. That seems less than ideal to me. To counter that, there could be some kind of runtime service that handles the effects from importing a module. Suppose some user code wants to mess with the effect handling, though?

I don't know if I'm satisfied with this idea, but maybe if there were some kind of sandboxed execution facility, so that someone who wanted custom effect handling could "simply" create an object to emulate the runtime, and within that emulated runtime, imports would be done completely freshly.

I'm tempted to see if the syntax could be made to support Lua's function call syntax, so that there's just a function instead of a statement. If this were a function, it would effectively only propagate errors and non-termination to the calling code, and all other effects would "pass through" the intervening layers to the runtime.

Final question to consider for tonight: does the fact that call trees all trace back to the __main__ module, combined with the ideas I've been going over, come together in a way that's elegant, inelegant, or pointless?

Well, let's look at the Hello World program for NABTO. It's going to be something like println("Hello World!") (probably with some boilerplate at the beginning). The runtime will import the module, and the import system will handle the text output effect. Then, it will finish with the module, and exit.

A more complex program will end up importing multiple modules, and suspending execution during the imports, then resuming, possibly with an error.

This all seems to make sense to me, but I really should sleep on it before I think I'm sure.

Good night.

Coding 2022-08-25

2022-08-25T04:00:00-04:00

I figured I'd keep on thinking about the programming language concept I sort of mentioned last night. Before I do anything else, I want to name it so I can have something convenient to refer to, but I don't want to commit to a name yet. As such, I'll use a working title that practically demands to be replaced, once a suitable replacement suggests itself:

Name Already Been Thought Of, or NABTO. This is an allusion to trying to come up with some punchy, memorable name, and oops, it's already in use. With its somewhat unusual combination of different points of articulation and voicing, NABTO is the programmer art of names, which is what I need, psychologically, to get on with it.

So, one of the major inspirations for NABTO is Python. I use Python a lot, and I wonder what it would be like to use a language that is like Python, but, at a few points, does things the way I would do them, seeing how they worked out in Python.

Two big pain points for me in Python are inheritance and __init__ methods. As of the last few years, when I'm left to my own devices, I use inheritance rarely, and explicit __init__ methods less. Here are the statistics from the current MOTR codebase:

Classes that inherit:
- From typing.Generic: 20
- From typing.Protocol: 16
- From enum.Enum: 3
- Internally: 9
- From third-party code: 3
- From an exception: 2
- Not at all: 41 (+2 required to exist by third-party code, but with no inheritance)
Explicit definitions of __init__: 0 (thank you, attrs)

So, from looking over these, we have 39 instances of inheriting from the standard library to get specific functionality, most of the internal ones and the exceptions are cases of ontological inheritance, and the third-party stuff is to hook into third-party functionality by inheriting a bunch of functionality.

Ontological inheritance is one of three kinds laid out in this post. The others are abstract data type inheritance and implementation inheritance. For my purposes, I'm going to separately consider inheriting the implementation from a concrete class and inheriting the implementation from an abstract class. I like this distinction because I generally distrust the former, and I apparently do the latter a lot.

So, if I want NABTO to reflect my own ideas about how to code, how do these different types of inheritance look?

Suppose we consider different types of inheritance to be associated with different types of type.

We can have concrete types that represent some data in memory and a set of associated behaviors. We can have ontological types that represent open or closed sets of values. We can have abstract data types that represent the contracts that a value upholds. We can have mixin types that provide implementations of behavior without reference to state.

As to how these types can relate to each other:

concrete types can have their instances point to other values, which will also be instances of concrete types.
concrete types can be added to open ontological types; multiple inheritance imposes no constraints on ordering.
concrete types can implement the contracts of abstract data types; multiple inheritance imposes no constraints on ordering.
concrete types can inherit from mixin types; multiple inheritance imposes constraints on ordering, and I think the safe bet is C3.
ontological types can relate to each other, and I'm not sure they should relate to anything else?
abstract data types can require that their operations take values of specific concrete types, or that conform to other abstract data types, but I don't know if it makes sense for them to care about ontological types or mixin types; they can also inherit from each other to express relations like "a sequence is also a container" or whatever.
mixin types can inherit from each other, and their methods can specify concrete types, or ideally abstract data types, in their signatures.

Right, signatures. I kind of want to see how gradual typing looks when it isn't bolted on after-the-fact. Maybe there are languages out there that are trying that, but after all of the languages I name-checked yesterday, I feel a little researched-out.

That off-hand reference I made to open or closed ontological types earlier made me realize something. Exception handlers in Python are like match statements that only work on instances of a particular type, passing through specific mechanisms. I'm really interested in Koka's ideas about minimalism; seeing them in action caused some ideas that have been kicking around in my head for over a decade to make sense, so let's see if anything inspires me there...

First off, I had some ideas for using one type like another type. Like, concrete types and abstract data types could be used as ontological types for some purposes, and a concrete type could be used as an abstract data type. My gut feeling is that using concrete types like that is likely to be a problem a lot of the time, but I think experience is worth more than feelings, so I think it should be allowed, but warned against. That way, if someone really needs it, they've got it.

(This sort of clarifies some of the uses of concrete types, above. Actually, it's a concrete type being interpreted as an abstract data type.)

It's getting late, and this post is apparently quite long, so I'm just going to spitball some thoughts about how different types of types could interact with match constructs.

Only concrete types could perform destructuring match, since the other types don't specify a structure. I haven't gone over the aspects of concrete types that should make destructuring possible.
Ontological types could check for membership, which is straightforward enough.
I'm not clear on what abstract data types or mixin types could do in the context of a match.

After thinking about this a little more, my feeling is that concrete types should handle their own destructuring, so the language- or library- level matches should deal with ontological types.

In any case, this doesn't quite feel fully baked, but I want to be done with writing tonight. I think what I need to do is put together a Sphinx project for documenting how I think this all should work, before I try to prototype it.

Good night.

Coding 2022-08-24

2022-08-24T04:00:00-04:00

I figured I'd talk some about something that's been on my mind, something that got me looking into Koka. It's this post. There are a bunch of interesting ideas in there, and I'm interested in seeing, more or less, what happens if I aim for a point in the design space somewhere in the convex hull of this post, Koka, Python, and the CLOS (n.b. I do not know Common Lisp, so I'm trying to get this through reading The Art of the Metaobject Protocol).

There are a lot of fiddly points to consider. Like, should there be a Python-style descriptor protocol? Koka (and Perl, I think, and presumably other things) style nullary functions with implicit parens? Both, somehow?

But I want to focus on one bit from Eevee's post, where she's talking about Python, and I'm not sure she's correct.

Let's quote the relevant bit.

Magic methods work differently from other methods, in that they only work when assigned to the class and not when assigned to an instance. It turns out there’s not actually a good reason for this.

This is going to need a bit of unpacking.

To start with, let's talk specifically about infix operators, like +. Haskell and Koka implement infix operators with a bunch of syntactic sugar, that allows the representation of the operator to be named in prefix form. So, if you crack open the definition of, say, list concatenation, you get

(++) []     ys = ys
(++) (x:xs) ys = x : xs ++ ys

// Append two lists.
pub fun (++)(xs : list<a>, ys : list<a> ) : list<a>
  append(xs,ys)

// Append two lists.
pub fun append(xs : list<a>, ys : list<a> ) : list<a>
  match xs
    Cons(x,xx) -> Cons(x,append(xx,ys))
    Nil -> ys

Unlike Haskell or Koka, Python implements "define an implementation of an infix operator" by having the class in question define a method named with __double_underscores__, like + with __add__. (And __radd__, and there's also __iadd__, which should do something similar to __add__ but not the same, and...)

Python also uses this kind of mechanism for unary operations, and various other bits of built-in functions and syntax. Now, what "works" here is the mapping between "fancy syntax" and "the actual method implementation". If you have

my_var + 3

then executing that code is going to result in the __add__ method being looked up on the type of my_var, and passed my_var and 3. You can get similar behavior by invoking the method directly on the instance, although this is brittle in certain ways:

assert my_var.__add__(3) == my_var + 3

(If we used a user-defined type for the right-hand side, this would not necessarily hold. This is getting into that __radd__ stuff from earlier.)

What Eevee is pondering is the equivalent of

my_var.__add__ = custom_add_function
assert my_var.__add__(3) == my_var + 3

This is not likely to work, because Python doesn't look up the "magic" method on the instance at all.

As to why I'm not currently comfortable with the statement that "there's not actually a good reason for this", we'll have to look at the reasons that I do see for it.

If I'm understanding my experiments correctly, attribute access in Python essentially "looks" in two different directions.

On an instance that is not a type, instance.attr will check the instance dictionary for attr, and the instance dictionary of its type, and its type's ancestors. The behavior is eerily configurable if you're determined, and the default behavior is slightly more involved than I feel like summarizing currently.

On an instance that is a type, the my_type.attr situation is similar, except that now the instance has type ancestors to contend with, as well as the ancestors of its types. (The type of a type should be a subtype of the type of its supertypes, so there's no problem only checking the type of the instance itself.)

Where things get hairy is not necessarily __add__, but one of the more commonly invoked operations of a type.

Suppose we want to create a type whose instances can be called like a function. This looks like

class MyClass:
    def __call__(self):
        ...

Now, one type whose instances are often called like a function is... type.

>>> class TalkyMeta(type):
...     def __call__(self, *args, **kwargs):
...             print("Hi there!")
...             return super().__call__(*args, **kwargs)
...
>>> class Talky(metaclass=TalkyMeta):
...     pass
...
>>> Talky()
Hi there!
<__main__.Talky object at 0x7fd4850de110>

As long as we have types acting as the main entry point for constructing their instances, using the __call__ method, then we can't look up all "magic" methods on the instance without making it highly inconvenient to construct an instance of a type that overrides __call__ so its instances can emulate functions.

For what it's worth, Lua's metamethods avoid taking a stand on this by not allowing the instance to access the metamethods via instancing unless you specifically hook it up like that. There's one table for metamethods, and another table for extra attributes visible on the instance, and maybe they're the same table, but they don't have to be.

It's getting late, so I'll just say that my gut feeling is that, instead of trying to make the magic method lookup take the instance dictionary into account, it should (somehow) get its own mapping on the type, distinct from the "stuff that the instance delegates to explicitly".

This raises a bunch of questions in the context of language design, so this idea is way more of a starting point than an ending point, so there's plenty more for me to think about, later.

Good night.

Weekly Roundup 2022-08-23

2022-08-23T04:00:00-04:00

Wednesday: I looked into the blog's mobile usability issues, since apparently the blog is in the mobile crawler bucket.
Thursday: I did a bunch of work on the draft changes to the templates and stylesheet, and virtually no work on the post.
Friday: I made a bunch of progress on the redesign.
Saturday: I didn't get too much done, because I was watching a baseball game on Twitch.
Sunday: Some work done, but it didn't really feel like much.
Monday: I basically finished my stretch goals for the redesign.

Next week, I'm going to take a few days off from the redesign, check over it, and roll it out, let's say, over the weekend. Ish. I've got other stuff I want to mess around with in the meantime, but I'm not sure how much I'm up for, since I was sick yesterday and didn't fully realize it.

Site Design 2022-08-22

2022-08-22T04:00:00-04:00

I've been tweaking things, and now I think I just need some time to consider any further changes I may want to change.

When I tested out the Bonus Content, I ran into some issues that imply to me that I need to remake it. Fortunately, it's just the assets, all of the code and styling is done. I might have time to redo this after this post, but I don't want to block this post on that.

I didn't get to the Python code I wanted to update, but oh well. Not sure about next weekend, but we'll see.

Anyway, I want to wrap up for now.

Good night.

Diary 2022-08-21

2022-08-21T04:00:00-04:00

All right, the blog code is handling the new directive for the Bonus Content. I still need to create the theming to go along with it, and make sure I can properly integrate it with the stylesheet, but at least I know I won't trip over my shoelaces at the finish line.

I finished my stretch goals for the tooling stuff for my other writing today, so that's good.

And I took some design notes on the Bonus Content.

Sadly, I didn't have time to work on the Python code or the writing, but I'll at least try to lay the groundwork for the writing tomorrow. I just created the project file that I'm going to import the current draft into. That's a small step, and I could do it at any time, so why not now, I figure.

That doesn't apply to anything else, because I have like half an hour to midnight, and I'd rather not spend all of it on these entries.

Basic tasks for tomorrow:

Refresh my memory on the Python code, maybe start fixing stuff up.
Sketch concrete designs for the Bonus Content.
Get on with importing the draft.

The big thing I need to handle that is to try and actually do stuff in the morning. For now, I'm going to get ready for bed.

Good night.

Diary 2022-08-20

2022-08-20T04:00:00-04:00

Okay, so, this isn't going to be a long entry because I didn't have time to come up with anything. I like that things are coming together with the site design, but I've got other stuff I want to be working on.

I think my best bet there is to get to bed as soon as I can, then make sure I have a properly relaxing morning. We'll see how that plan works out for me.

Basically, I want to take care of the following things:

Art design for the remaining parts of the site rework.
Code support for the remaining parts of the site rework.
Final tweaks to some of the support tooling for my other writing.
Begin the next editing pass on the other writing.
Juggle python projects.

Yeah, this seems somewhat... ambitious. But we'll see tomorrow. For now, I need to sleep.

Good night.

Site Design 2022-08-19

2022-08-19T04:00:00-04:00

Things are coming together on the redesign. More accent colors showing things off and spacing things out, better scrolling behavior, local fonts so my fonts get served from the same CDN as my content, tags overhauled to show up properly...

By the way, I don't mind if anyone tries to copy stuff from this site's CSS, either before or after I roll out the update, but if it does occur to you, keep in mind that I'm self-taught, and using trial and error on a small set of browser configurations. While I believe the style and template updates will represent a net improvement in presentation quality, there could be all kinds of regressions or unaddressed issues.

I want to take a few days to poke around for additional issues, and work on the Sweet Bonus Feature that I was prioritizing everything else over. At some point, I should maybe try to put together some kind of automated tool for evaluating drafts of the blog without having to click around myself to find every issue like "Oh no, this rule is applying too much", or "this element still has the wrong width if the screen is too narrow" or "there are too many scroll bars".

Anyway, yeah, a better phone experience is coming. At least, assuming that "smush it into a smaller viewport than apparently anything on the market, and assume it works at all intermediate resolutions" is valid enough as a way of improving the phone experience.

Good night.

Site Design 2022-08-18

2022-08-18T04:00:00-04:00

I made a bunch of improvements or changes to the site layout and templates on a branch, and I'll hopefully be able to roll it out soon.

I spent too long working on that to have anything more substantive to write in this entry byyyyye

Good night.

Site Design 2022-08-17

2022-08-17T04:00:00-04:00

While I was thinking about stuff to do, I remembered that Google was yelling at me about mobile usability. (I don't know, the blog works fine on my phone...)

Anyway, that got me thinking about updates to make to the site.

For myself, I want to look into a few possibilities:

Make the main content area wider, mainly to fit more code on the screen when I'm messing with code blocks. So, my maximum goal there is 76 characters.
Add previous/next controls to the top of pages that only have them at the bottom (and vice versa, if applicable)
SECRET BONUS STUFF
Maybe I can try to get tags working again?

Google is raising a bunch of usability issues, but it's not raising them about any pages currently, and it only raises them about a few pages at a time, which... That's weird, right? I'm not doing anything radically different on different pages that would result in exactly seven pages having issues with:

Small text (it can't be complaining about the main body text, right? That's 18px)
Clickable elements too close together
Viewport not set

I guess I'm going to have to research viewport stuff. Like, what's the viewport to set for "it's a static blog, I don't care"?

From further messing-around, I haven't gotten it to pin down a specific broken page, so I'm going to ignore the text and clickable elements for now, and look into the viewport, because I could believe that's an issue even if it doesn't show up when I have it test a page (???).

In any case, I'm going to finish up and publish this post, then queue up some reading for later on responsive web design, because I guess I need to look into that.

Good night.

Weekly Roundup 2022-08-16

2022-08-16T04:00:00-04:00

Wednesday: I planned out some stuff for generating HTML with Koka.
Thursday: I went a little further with that, speculating about various representations that would make sense at different stages.
Friday: I did work on the HTML stuff and other stuff, mostly the HTML stuff.
Saturday: I was wiped out from traveling.
Sunday: I revisited some of the code that MOTR is for, and updated it.
Monday: I was wiped out from traveling again, so I decided to mess with a new project that I'd picked up over the weekend.

Next week, I don't know, I'm not quite up for planning I guess. Try to get spun back up with Koka, maybe.

Coding 2022-08-15

2022-08-15T04:00:00-04:00

Uuuugh...

We were wiped out by travel today, so I'm going to just focus on the graph stuff I was working on earlier.

Basically, I don't have a specific usage in mind for this, but I feel like it'd be interesting to have a Graph[T] data type that can hold many different types of data (so it's functionally going to end up as a Graph[T | U | V | W | ...], and I admit I'm not sure if I can strike a good balance between useful interface and type safety here...), that takes typed edges to move between vertices. (The likely endpoint for this, if it makes any significant progress, is "not-very-good database".)

One thing that I wanted was to have a distinction between "edges added explicitly" and "edges calculated from other edges". (And potentially, some of these "calculated" edges could have the ability to act as a shortcut for adding explicit edges, provided there's a unique minimal graph satisfying "this edge goes between these points".) Now, something that I hadn't fully worked out until recently, and I think it's going to be significant, is that "calculated" edges can't just produce an arbitrary value; it has to be an actual vertex in the graph, and the only way to guarantee that is to only work with the graph structure, and ignore the actual contents of the vertices.

The thing that I'm trying to figure out right now is, when does it make sense to talk about following an edge backwards? Explicit edges can always be followed backwards.

Let's do a hand-wavey induction thing and consider, supposing we have some collection of "reversible" edges, how can they be composed?

Basically, an edge takes a vertex in a graph to a set of other vertices, so "calculated" edges need to just combine and work with these sets.

We can produce an edge by intersecting the sets of two edges, and reverse that edge by intersecting their reversals; similarly for unions.
We can also compose two edges and union the resulting set of sets; I'm not sure that intersecting the set of sets makes sense. ... Rather, it's that I think the union reverses cleanly, and the intersection doesn't. With the union, you're basically saying "this edge, then this edge". With the intersection, I was going to try to work through some stuff, but then I realized, wait; you can make the intersection of the composition shrink by adding edges. That's wrong, I think.
We can extend an existing edge with a predicate function that takes two variables; this does allow us to reach into the vertex data, but only for the purpose of choosing a course of action regarding vertices that exist.
Now, maybe there's some way to extend that concept, because I know I want something like:
- An edge that exists if underlying edge E1 exists
- But only if the target vertex has an edge E2 to vertex V.
- This is fundamentally just a predicate that requires the graph in order to define it, so let's just throw another variable in there.

I realized the stuff in that list, not quite in the order that I placed it up there, so now I've got two things:

Confidence that every edge type I've thought of should be reversible.
Knowledge that I have to be careful with the graph-based predicate idea. Like, that's going to break if it's possible to express the idea of "express this edge if this other edge doesn't exist".

An incomplete list of things that should probably work with that:

Reasoning about the unions of arbitrary sets
Reasoning about the intersections of predefined sets
Filtering a set based on a predicate that doesn't refer to the graph

A lot of this graph stuff is basically a recreation of the "calculated" edge stuff, which implies that it should be possible to express these "graph" predicates as edges, and instead of "three-argument predicate", have something that's just "if this edge exists, and this other edge leads from the [start/end] of that edge, to at least one other vertex". (Maybe allow reasoning about cardinality in order to be fancy, but it's not obvious to me why the threshold should ever be higher than 1.)

This has been some good rubber-ducking for a project that I don't have any idea what I'm going to do with it, and frankly, I'll take that.

Good night.

Coding 2022-08-14

2022-08-14T04:00:00-04:00

I'm still kind of out of it, so instead of holding myself to grinding out some improvements to MOTR, I had a look at one of the projects I intend to use it with. The project had suffered some bitrot, so I took the time to update the things that had changed.

Basically:

It was using my old fork of flake8-html, so I pushed my local copy to GitHub, and pointed both MOTR and this project's requirements files to the branch, so that other people could, in theory, run the test suites and see the failures for themselves. I also updated the fork with the latest commits from master. And pinned the flake8 version for now.
Mypy got some good updates that I had to update some of my code to deal with. The result probably makes more sense.
The shiv build step included an argument being passed to pip that broke the build because it wasn't recognized. So I removed it, and it seems fine? I wonder why I had it in the first place... Ah, looks like I was checking for any fallout from a then-upcoming change to pip, that went in, so they took out the option to enable it.

Anyway, one reason I'd been looking at this repo is that it hosts my punq fork (which I'm going to rename, for sure). The fork relies on the behavior of PEP 563 to construct nicely namespaced types in a registry for dependency injection. My impression of the plans for annotations is "switch to PEP 649, once we figure out how", so I figured I'd investigate things in this area. I'm aware that the switch will break the punq fork in fascinating ways (and I'm not totally clear on how original punq will handle it when it comes around), but my impression from looking at some of this code, and thinking about the rest, is that I can probably just get rid of a bunch of non-standard behavior, replace it with NewTypes, and delete a whole bunch of code that I barely remember writing, but that I know for a fact isn't anybody else's fault.

I'm thinking I'll mostly write this weekend off and mess around with other stuff, but once I've got my energy back, I think it'd make sense to alternate between updating MOTR and this other repository. Like, make changes to one repository, run tests, spend a few minutes on the other, run tests, go back to check the first set of tests...

That seems like it makes sense. Next week, or later, I'll see how it shakes out in practice.

I've got a bit more time tonight, so I'm going to mess around with something else for now, to blow off steam.

Good night.

Diary 2022-08-13

2022-08-13T04:00:00-04:00

We were on the road for a while, so I'm not really capable of focusing on stuff. My short-term plan is to write this entry, then scribble some ideas on paper.

There are a bunch of ideas swirling around in my head, and I need to take some time to crystallize them, and hopefully I'll be able to pursue them later, either in Python once I have MOTR in an acceptable condition, or in some other language if it'd be a better fit.

Stuff about graph data types, and what inheritance should accomplish in a programming language idea I'm messing with. And probably other things that I'm not remembering offhand. There's no direct relationship between any of these ideas, but I'm thinking about them at the same time.

Good night.

Learning Koka 2022-08-12

2022-08-12T04:00:00-04:00

I'm having a kind of distracting few days, so I might not get much written for a few entries.

I did mess with some other stuff in Koka earlier today, but I didn't have much luck; it looks like some of the generics stuff I wanted to do is not super obvious in the documentation, if it's even possible.

Anyway, I'll write some quick obvious code for the HTML stuff.

Hm, wasn't as quick as I thought, and also I found some limitations in the vim plugin for Koka, which isn't a surprise, since I think it's a third-party thing.

I don't want to stay up overlong, so I'm going to start winding down now.

Good night.

Learning Koka 2022-08-11

2022-08-11T04:00:00-04:00

I'm going to stick to the HTML stuff for now, rather than switching with the structuring concurrency.

I have a feeling that I'm doing something kind of imprecise with my mental models of this tag rendering stuff, so let's see what happens if I try to lay down some details.

Tag(
  "html",
  [],
  Just(
    (
      "",
      [
        (Tag("head", [], Just(("", []))), ""),
        (Tag("body", [], Just(("", []))), ""),
      ]
    )
  )
)

Which should turn into

<html><head></head><body></body></html>

Obviously, the Koka code is not how stuff should be written out manually...

And to tweak the output a bit...

Tag(
  "html",
  [],
  Just(
    (
      "",
      [
        (Tag("head", [], Just(("\n", []))), ""),
        (Tag("body", [], Just(("\n", []))), ""),
      ]
    )
  )
)

Which should produce

<html>
    <head>
    </head>
    <body>
    </body>
</html>

How it gets there doesn't feel straightforward to me. Like, maybe there can be a list of lines, where each line is a list of elements, and once the line includes a line break, it gets split up into a bunch of lines?

So there's then another representation like

[
  [
    StartTag("html", []),
    StartTag("head", []),
    EndTag("head"),
    StartTag("body", []),
    EndTag("body"),
    EndTag("html"),
  ],
]

I've drafted out basic code for representing this stuff, but I should figure out how to represent stuff like "tags that don't reflow their content in a templating context".

I'll have to think about that some, maybe switch gears again later. For now, I'm tired.

Good night.

Learning Koka 2022-08-10

2022-08-10T04:00:00-04:00

I thought about stuff to do with Koka, and I ended up deciding to try to focus on one subsystem from the hypothetical CGI idea: just generating HTML.

The basic thing I'm interested in trying to do, is to make the generated HTML indent nicely. If you ask around online, people will tell you this is "not important" and such, but I'm looking for something that interests me, rather than that addresses any burning need.

I'm thinking some kind of library of functions that lets you write all of this as code. Have functions for each tag. So, what we need to know about how this stuff all works?

Well, there's the doctype thing. Some tags can only appear in specific contexts, which it might make sense to express in the type system. For now, I'll just think about that in terms of the <html>, <head>, and <body> tags, since those are simple.

Maybe try to separately address "this is how to convert this tag object to a string" and "this is how tag objects are allowed to relate to each other".

Like, a generic representation of "HTML tags, but ignoring all semantics"... That looks something like:

a tag name
an alist of attribute-value pairs
maybe<> a structure that represents the tag contents, which can be, say, maybe<> a string, followed by a vector or list of tag-text pairs.

I believe a few tags would need special-casing (I'm thinking <pre> and <script> at least), but the majority should be able to work from just that.

Anyway, I let this get late. I'll actually try to code stuff later.

Good night.

Weekly Roundup 2022-08-09

2022-08-09T04:00:00-04:00

Wednesday: I tried to explain what interests me about Koka.
Thursday: I talked about using Koka for a specific practical-ish application.
Friday: I speculated about what a structured concurrency library in Koka would look like.
Saturday: I did some more work on CGI in Koka, and ran into bizarre issues that kind of make me want to come up with some other project for Koka, and I'll mess with rolling my own CGI library in Lua or whatever.
Sunday: I started reviewing MOTR's codebase to look for obvious problems, and sorted them in rough order of when I want to be working on them.
Monday: I finished the review, though I'm going to need a few more passes. I also made some of the changes, and got to see how justified they were because I was squinting at some lines going "Wait, which of these attribute accesses is broken?"

Next week, I guess I'll look for some other project to switch off with for the structured concurrency stuff.

Coding 2022-08-08

2022-08-08T04:00:00-04:00

Starting again, picking up from yesterday...

Change ASAP
- Names
  - CommandBuilder needs a different name for the command attribute, which needs a new name for its type.
  - In fact, I believe the type should be renamed Executable, after the fact that the x bit is set on the file, and its command attribute should be name instead. The same file has some type aliases that derive from the type, and they should be updated as well. Consider renaming Metadata, mainly because it's not a very descriptive name. It might fare better if I moved it to the command_builder module. Consider that.
  - The entry module seems reasonable, but come back to it later.
  - The flex module feels like it needs an overhaul, but I'm not sure what to replace anything with. Changes may propagate to the parametric module.
  - A lot of the class names in input_accumulator are kind of awkward, but I don't have a clear idea of what to replace them with, because I think I need to figure out more about the usage. All the same, it's worth seeing if the decorators can be turned into classmethods.
  - In installers, I think maybe Environment should be ExecutionEnvironment. I'm not really sure, though.
  - Not sure if the parameters module needs anything.
  - A bunch of the functions in the parametric module could be converted to methods. I'm not sure about "should"...
  - motr._api.cli_types.registry could do with some renames. Like, the point of it is just that these are the constants, maybe, to go with the parameters.
  - The typed_mapping module and protocols could also do with a rename.
  - motr._api.cli.build could do with the same classmethod rewrite I'm considering elsewhere, if I go through with it.
  - The cli modules have the general thing of, I should probably combine the python versions adaptor and the pip arguments label into a single object of some kind. And there are other changes I want to make, that I've laid out elsewhere. I'm skimming over this stuff, so I'm going to have to go into more detail again later.
  - The python_helpers module needs the requirement_invocation function to be renamed. Not sure about anything else.
  - The reports module could have the acumulator function renamed, or not. We'll see.
  - Don't know about the pip module, name-wise. Or otherwise.
  - The package module has some functions that I should see if I can get rid of.
- Everything else
  - isinstance() calls in the command module.
  - I'm considering dropping many uses of PVector[T], since I'm not really, like, using any part of the interface beyond what I'd get from a Tuple[T, ...].
  - isinstance() calls in the parametric_command module. Also, there are some types that could be improved with a typed mapping.
  - reduce_parameterization() could have the arguments reorganized a little so that reducer and initial_value are in a single container object.
  - Some of the internal strings in motr._api.cli.build are wrong, and need to be fixed before the next release.
Change after the next release
- Names
  - The cmd_ function is somewhat unfortunate. Consider converting to a classmethod?
  - I'm a little iffy on whether to keep the Input and Output names. They could be Prerequisite and Outcome, for example, but those are mega long. This would spill over into the not_output module.
  - mkdir and make_parent could be moved if cmd_ is, for consistency.
  - Same for write_bytes.
  - I think there could be a better name for motr._api.build and its build() function, but I think to find it, I'd also need to find a better name for motr.core.registry. Which, to be fair, it definitely needs a better name.
  - I've got a note to rename motr._api.requirements.action to task. This would go along with some changes to core.
  - Similarly, name_target to name_outcome.
  - About requirements overall... I feel like something should change, but I've got time to figure things out.
  - And, target to outcome.
  - I've got a rename queued up for motr.core.target_name as well.
  - And motr.core.target.
- Everything else
  - The motr._api.actions modules, as well as motr._api.cli.build need an overhaul to their interfaces, which is going to start with changes in motr.core.runner. Basically, I want to change from "return a mapping of output when done" to "pass in some kind of callback for registering output streams".
  - I want to beef up the details of some of the typed_mapping stuff, and also I want to spin it out into its own package. Making it into a package would allow me to use it in other projects, and also make the test runs faster in MOTR.
  - I'm not sure if this counts as "name" or "everything else", but I'm not totally sold on the way I'm doing re-exports from the api module. Like, is this confusing to people who aren't me? I need to figure that out.
  - The base controller is huge and needs to be broken up. I don't know how, exactly, but it's too big.
  - I should see whether I want to add any classes to the exc module.
  - I've got a bunch of renames cued up in motr.core.registry. The thing I'd like to figure out is, what's a better name for all of this than "registry"? DependencyGraph? Let's go with that for now.
  - As I said, I want to tweak some things in motr.core.runner. Like, renaming RuntimeAction to Task, and Target and Action to, um... Apparently, Request, and, let's say... RequestTask.
  - motr.ext.ext_maybe_format I think needs to be broken up into multiple extensions, but we'll see.
  - Paired changes to motr.main and motr.motr_app around disabling Cement's ctrl-c signal handling so Trio can have it.
  - The line CONFIG["motr"]["foo"] = "bar" needs to go.

Well, that's a good start for noting this stuff down. I did this earlier in the day, so maybe I'll feel like making some of these changes later. We'll see.

Okay, it's later, and I did some updates. Things are better, but there's more to be done.

Good night.

Coding 2022-08-07

2022-08-07T04:00:00-04:00

All right, let's start another pass through the MOTR code, to see what changes I want to make. I'm going to divide this into two pairs of categories: changes to make as soon as possible versus after the next release; and changes to names, and everything else. My plan is to hold off on "everything else" until I have full test coverage, rather than 94%.

But let's break this down:

ASAP
- Names
  - CommandBuilder needs a different name for the command attribute, which needs a new name for its type.
  - In fact, I believe the type should be renamed Executable, after the fact that the x bit is set on the file, and its command attribute should be name instead. The same file has some type aliases that derive from the type, and they should be updated as well. Consider renaming Metadata, mainly because it's not a very descriptive name. It might fare better if I moved it to the command_builder module. Consider that.
  - The entry module seems reasonable, but come back to it later.
  - The flex module feels like it needs an overhaul, but I'm not sure what to replace anything with. Changes may propagate to the parametric module.
  - A lot of the class names in input_accumulator are kind of awkward, but I don't have a clear idea of what to replace them with, because I think I need to figure out more about the usage. All the same, it's worth seeing if the decorators can be turned into classmethods.
  - In installers, I think maybe Environment should be ExecutionEnvironment. I'm not really sure, though.
  - Not sure if the parameters module needs anything.
  - A bunch of the functions in the parametric module could be converted to methods. I'm not sure about "should"...
- Everything else
  - isinstance() calls in the command module.
  - isinstance() calls in the parametric_command module. Also, there are some types that could be improved with a typed mapping.
  - reduce_parameterization() could have the arguments reorganized a little so that reducer and initial_value are in a single container object.
Post-next-release
- Names
  - The cmd_ function is somewhat unfortunate. Consider converting to a classmethod?
  - I'm a little iffy on whether to keep the Input and Output names. They could be Prerequisite and Outcome, for example, but those are mega long. This would spill over into the not_output module.
  - mkdir and make_parent could be moved if cmd_ is, for consistency.
  - Same for write_bytes.
  - I think there could be a better name for motr._api.build and its build() function, but I think to find it, I'd also need to find a better name for motr.core.registry. Which, to be fair, it definitely needs a better name.
- Everything else
  - I didn't get to this normally, but the line CONFIG["motr"]["foo"] = "bar" needs to go, for obvious reasons.

I got distracted with stuff today, so I didn't have time to get through all of this. Tomorrow, I'll pick back up from the parametric module.

It's nearly midnight.

Good night.

Learning Koka 2022-08-06

2022-08-06T04:00:00-04:00

Okay, I didn't get much done today, but I did sketch out some helper functions that seem to work, at least with the interactive compiler. I seem to be having some issues using them in compiled code; I'll have to look into that later.

Anyway, my basic plans here are to really firm up the environment variable processing, focus on the response headers after that, and then just start messing around with writing templates or some other form of document generation.

For now, though, I'm going to get to sleep and wonder what I'm doing wrong in my main() function.

Good night.

Learning Koka 2022-08-05

2022-08-05T04:00:00-04:00

So, I don't think I really explained this, but I'm sort of alternating what I'm focusing on in looking into in Koka. Yesterday, I was seeing how much of the script side of CGI could be implemented in Koka. Now, I'm sketching out how to handle structured concurrency.

So, let's see about that.

So far, I've got two distinct effects that I think need to be supported. One for "resume the current execution at some future point" or, in other words "schedule the resumption of the current code in the future". And one for "schedule an additional execution, and ensure that all such executions have completed at some future point".

These are the equivalents of await and nursery.start_soon() in Trio.

I poked around with this a little, and I am not sure I'm writing things correctly. I currently don't have an argument to the await function in the async effect, and I think it actually needs an argument to describe when to resume. Like, a higher-order function that takes the resume function as an argument. And some kind of queue to put them in, I guess...

Anyway, we'll see how much focus I have to put into this later. For now, I need to get to bed.

Good night.

Learning Koka 2022-08-04

2022-08-04T04:00:00-04:00

I'd mentioned yesterday that I want to think about more concrete problems before I mess with trying to do structured concurrency in Koka.

As to the specific thing I'm choosing, it's not a good fit, I don't think, but I'm interested in at least laying out why it's not. And what is it?

Okay, so, I never used CGI scripts directly. I used to see other people use them all over the place, but by the time I was working on web server stuff for work, I was doing things that used the WSGI. I don't know why, but I just decided, let's see how any of this actually worked.

So, I've got RFC 3875, and I'm looking over it and comparing what it wants to what's possible in Koka (assuming I don't write extensions).

Here's what I've concluded from what I've looked at so far:

CGI deals with standard input, standard output, and environment variables. The environment variables are expected to have a bunch of structure that the script should interpret before it performs any other I/O.
Koka should be entirely able to handle the standard output side of things. Writing CGI scripts in Koka would benefit from a library to generate HTML, and, I think, HTTP headers.
Koka should be able to handle the environment variables. The main thing I would need, is to have a case insensitive comparison for traversing the association list. (Which I later realized, after mentioning the concept earlier, that Koka's standard library does expect and support.) And then a bunch of processing steps.
I think the big stumbling block is the CONTENT_LENGTH variable. This is in octets, which is what you call bytes if you don't want to get well-actually-ed. The RFC states "the script MUST NOT attempt to read more than CONTENT_LENGTH bytes". Right now, Koka's facilities for reading from standard input are... limited. The standard library provides the readline() function, which reads from standard input, in UTF-8, either a line of text (discarding the newline), or 1023 code points, whichever is shorter; I'm not clear on what happens if it doesn't get an explicit end to the line within 1023 code points. In any case, the point is, CGI really wants the script to be working in bytes at the boundary, and I don't think Koka is up for that yet. I don't know if supporting this kind of thing is on the radar.

So, Koka, which is, keep in mind, experimental and under development, appears to not be ready for the entirety of a use case that it's not explicitly meant to support. I think that's fine for now, and I'd like to mess with the areas that it can support. If it works well, maybe I'll end up with a little static site generator. I don't see such a thing replacing Pelican for me, but it could be neat.

Anyway, I didn't exactly use my time super well today, so I'm not going to go much further for now. I'll try to break things down in a little more detail later.

Good night.

Learning Koka 2022-08-03

2022-08-03T04:00:00-04:00

So, Koka. Why am I interested in it? The main thing that's special about Koka vs the other languages I know, is its effect system. As far as I'm currently aware, the effect system does two-ish things. At least one and a half. It allows library writers to define new flow control concepts, and customize existing ones. This is because it's sort of like exceptions, but resumable, and, if you opt into the fancy code generation, resumable multiple times. The other thing is, unhandled effects are part of the signature for the function that's not handling them, so they can be used to mark side effects.

Now, what has me interested is, well...

pub fun map(xs : list<a>, f : a -> e b) : e list<b>
  match xs
    Cons(x,xx) -> Cons(f(x), xx.map(f))
    Nil -> Nil

The precise details of the code aren't too important, what's interesting is the signature. map takes a list of objects of some type, applies a function to each element, and produces a list of the results. So far, so normal. But the other thing going on here is the e, representing the effects of the input function f. Those effects are passed along and emitted by the call to map.

So, if you wrap a map call in a handler for a custom effect that is emitted by f... You should be able to use the same library function to map using a function that, say, does cooperative multitasking, as not. Basically, what I'm seeing here, is higher-order functions that are transparent to function color.

Koka's an experimental language, and that's the experiment I'm interested in. I want to see what happens if you put structured concurrency's new primitive into a language that's designed to be tiny and extensible, and make a bunch of decisions differently from Python. (I am not the first person who's been curious what this would be like.)

Before I design too much of that, I want to get some more experience with Koka, so my plan is to mess around with Koka trying to solve a somewhat more concrete problem. More on that tomorrow...

Good night.

Weekly Roundup 2022-08-02

2022-08-02T04:00:00-04:00

Wednesday: It was a slow day, but I did what I could.
Thursday: I got some stuff done, but also I got really distracted.
Friday: I got a bit more done, and started turning my distraction into blog posts.
Saturday: I messed around a little with Koka, and didn't quite get anywhere impressive.
Sunday: I did some planning for improvements to MOTR.
Monday: I executed on the planning of the improvements to MOTR, and I only had one of the functions do the exact opposite of what it was supposed to once!

Next week, I'm going to take a break from writing, and focus on process improvements there. I don't know that I'll blog about that, so I might end up mixing things up a little as far as what categories I use during the week.

Coding 2022-08-01

2022-08-01T04:00:00-04:00

I wrote up the helper methods for MOTR, and after some slight implementation issues, I got them added to one of the tests.

Then, I got back into documenting stuff. I think it's time for me to start reconsidering the names of some of this stuff, because some of this code is hard to document when the names clearly have nothing to do with the purpose.

So, that's going to be the next stretch of work I do with this code: collect all of my misgivings about class and function names into a single place so I can prioritize them. I may also take notes on some functionality changes I want: stuff like removing isinstance() calls and streamlining the command wrappers.

Anyway, I'm exhausted for some reason, so I guess I'm getting to bed now.

Good night.

Coding 2022-07-31

2022-07-31T04:00:00-04:00

Okay, I did a bit more documentation just now, but the other thing I want to work on in tandem, is to set up diagnostic methods on the core.registry.Registry class.

Basically, let's see...

"Does every action have an output?"
"Is every action transitively reachable from a named output?"

Or, more detailedly, "Which actions do not satisfy etc etc"

I can put in more work orders to my future self, but I believe those two would feed into the tests I want to write, and would be a good starting point for further improvements. Best to put in minimal effort to the code itself, until I find specific deficiencies to address.

Anyway, I wanted to draft that tonight, but it's late, and I should get to this tomorrow.

Good night.

Learning Koka 2022-07-30

2022-07-30T04:00:00-04:00

Hey, guess what I remembered I could do!

Anyway, I messed around with some advice on "first programs to write in any language". The second thing it had me do was code up a subtraction game and an AI for it. It took me a long time to get it to compile, but once it compiled, everything worked perfectly and it beat me.

My impressions from getting it working were that using var and while together isn't worth it, and I ended up avoiding both by just rewriting everything as tail calls. I think I'm going to need to experiment with var, and later while, and try to get a handle on when to use them. Lastly, it seems to be better to write small functions, so they rack up fewer effects and it's easier to see where they're coming from.

I bet I could also make the code a bit nicer by cutting down on the mutual tail recursion, and coordinating stuff at a higher level, but I'm not going to try to figure that out right now.

One thing I'm interesting in figuring out is "the right" way to handle mapping objects. My gut feeling is that, if I try this, I should see how far I can get with association lists, unless I missed something that already does mapping.

Anyway, I'm wiped out and I'm going to try to get to bed earlier.

Good night.

Diary 2022-07-29

2022-07-29T04:00:00-04:00

Writing is going well. I've gotten to the point where there's a little bit that I feel like I need to fill in for the current draft, and besides that, it's just as much as I feel like adding on. Then I'll take a break, because I'm pretty sure I can't just grind away at this constantly.

Anyway, I mentioned some programming language stuff yesterday. I've been looking at Koka recently. It's an experimental language that is built around the concept of an "effect system", which is basically a generalization of exceptions, coroutines, output, non-termination...

It's a pretty ambitious idea, basically, and I'm curious to see what I can do with it; I've got some somewhat ambitious ideas in mind, so I'm trying to figure out a good limited-scope project to start teaching myself how to use it first. I might try out some of the ideas I had for messing with BQN... One thing I need to clear up for myself first, is when to be using a linked list, and when to be using a vector.

Anyway, I had plenty of time to wind down when I started this entry, but I went really slowly, so I'm going to just stop abruptly.

Good night.

Diary 2022-07-28

2022-07-28T04:00:00-04:00

Writing is still a bit slow. Partly this is because I'm trying to rewrite things in order, and I got to a tricky bit.

Partly this is because I was catastrophically tired all day and my ability to focus is effectively random. Like, I could try to push through this bit of writing. Oooor I could read the documentation of an experimental programming language.

Anyway, I'm not going to do any better if I stay up, so this entry is over now.

Good night.

Diary 2022-07-27

2022-07-27T04:00:00-04:00

I got a little writing done today. Very little. But some important reorganization happened.

Two things gave me trouble accomplishing much today. For one thing, I just had a lot of trouble focusing, and for another, I had a headache; I kind of still have it, in fact. I can feel parts of my teeth that I'm not used to feeling, and it is doing terrible things for my mood.

Hopefully I get to focus better tomorrow and accomplish more things. I shouldn't dawdle on getting to bed now, so let's wrap this up.

Good night.

Weekly Roundup 2022-07-26

2022-07-26T04:00:00-04:00

Wednesday: I got distracted writing stuff that was not the post, and barely wrote the post.
Thursday: I wrote a bit and made some plans for Friday, which is Saturday in this list because of The Reasons.
Friday: I had a rough day, and vented about it. Things subsequently got rougher, but at least those specific things I was venting about are basically done for now.
Saturday: The Gamera movies continue to be bonkers.
Sunday: I stared at MOTR's code, and worked out some relatively simple changes needed to make the tests pass. I also messed around with the tooling.
Monday: I made some progress on documentation.

Next week, I'm going to try to finish up this last stretch of revisions, then go over everything and make a list of things to revise before this is ready to show to anyone.

Coding 2022-07-25

2022-07-25T04:00:00-04:00

I'm adding basic documentation to the code. I also may be forking yet more code from tools that other people wrote. I'm going to have to work on contributing this stuff upstream.

Like, I've got:

flake8-html, but I made certain things display in multiple lines somehow, I forget
towncrier, but it works on Mercurial repositories
The pylint docparams extension, but it should properly respect no-docstring-rgx; that one is at least simple enough that I already described it on GitHub

(In my fork, I also attempted to make it parse the hecked-up docstring style I devised, but that didn't work because of mystery reasons. I think pylint's docstring parsing logic is maybe collapsing """valid""" inputs to invalid ones. Like omitting a section break from a Numpy style docstring.)

Anyway, I've got other stuff to take care of right now, so I'm going to wrap this up.

Good night.

Coding 2022-07-24

2022-07-24T04:00:00-04:00

Okay, I think I've addressed the issues with MOTR enough to make all of the tests pass. I've also managed to figure out a possible fix for some of the pylint issues I was seeing, but I want someone who knows the code in question to comment on my suggestions, in case I'm totally off-base. They'll get to it when they get to it.

For now, I've got a lot of documentation to update. I think for now, I'll tweak the configuration a little, and start documenting stuff tomorrow, maybe changing stuff tomorrow if the documentation inspires me. Hopefully restricting those changes to covered code.

Anyway, right now I need to get ready for bed.

Good night.

Diary 2022-07-23

2022-07-23T04:00:00-04:00

Well, I executed my plan for tonight nigh-flawlessly. I'm not feeling too talkative, so I'm going to finish up this post as soon as I can, and then play video games that will probably freak me way out.

I'm looking at Wikipedia and going, oh wow, I think there's still a few more Shōwa period Gamera movies that MST3K could cover, that's crazy.

Anyway, going to take it easy for the rest of today.

Good night.

Diary 2022-07-22

2022-07-22T04:00:00-04:00

Writing... I would have liked to have gotten further, but if I'm replacing any of the original draft, that's still progress.

I haven't really been talking about it here, but my personal life has been kind of rough lately. Nothing that's anyone's fault. I don't feel like it's been affecting the blog in any direct fashion, but my intuitions as to how a week of posting went have been sort of shaky, so I could be totally wrong.

I know there's plenty more the world could throw at me, but it feels like I deserve a respite. Like, world, see this year? Don't do that again.

I'm in a funk now, and I'm going to try to sleep it off.

Good night.

Diary 2022-07-21

2022-07-21T04:00:00-04:00

Closing in on a major milestone with writing, and there's not much more to say about that. I'll try to finish up the current segment tomorrow, and we'll see how that goes.

As far as Friday night... If there weren't any particular plan, I'd be trying to finally make some simple bug fixes to MOTR, but there's going to be an MST3K premiere, so that's going to be the main focus.

One thought about writing, actually... I'm planning to take a break from revising stuff soonish, and while I'm doing that, I'm going to think about changes to my various processes. So that'll be interesting, I guess.

Anyway, it's late and I shouldn't dawdle.

Good night.

Diary 2022-07-20

2022-07-20T04:00:00-04:00

I made some good progress on writing, but I ended up doing it until really late, and legitimately forgetting about this post.

I wanted to get in some work on MOTR, but I'm trying to keep that at a low priority during the week.

I can't think of anything else to say, and it's midnight, so I'll try to make things up tomorrow.

Good night.

Weekly Roundup 2022-07-19

2022-07-19T04:00:00-04:00

Wednesday: I did some writing.
Thursday: I worked through an annoying patch with writing, and planned stuff for MOTR.
Friday: The MOTR stuff concerned a design bug that turned out to be pretty subtle, so I'm trying to take things slow there.
Saturday: I couldn't write much because I needed to read, and before I work on the MOTR bug stuff, I want to work on documentation.
Sunday: I came up with some docstring syntax that Napoleon seems to process just fine, but so horrible that it looks like nobody tells you not to do it because why would they, but also none of the linters I've tried would accept it because why would they.
Monday: I set up two different parameterization doc linters, and settled on Pylint's for now because it seems more featureful, though seemingly a little buggy?

Next week, I'm going to grind through some more writing, and make changes and documentation to MOTR.

Coding 2022-07-18

2022-07-18T04:00:00-04:00

So, it turns out that Napoleon will happily parse my messed-up hybrid docstrings, but apparently every checker out there wants me to "use a single style" for some reason, so I'm trying out something that's close enough to numpy style for pylint to accept it.

Also, the motrfile is running pylint now. I shoved a lot of ignores into the rc, because the default checkers for pylint have... problems... As do the checkers I'm using on purpose, really. Pylint wasn't my first choice for checking my parameter documentation, but it seems to be doing better than my first choice on most metrics. I just need them to fix a few bugs for really real, and I can wait, since I have plenty of legitimate warnings to address.

My basic plan is to address all of the issues relevant to understanding the code that needs to change to fix the tests, then fix the tests, then fix the remaining legitimate issues, then disable pylint until they get some bug fixes in, maybe comment out some of the disables in the rc while I'm at it.

Anyway, I'm going to try to do some quick writing after I publish this. No more time for this post...

Oh also I guess I should add a pylint wrapper? But the relative ease I had just shoving the new code into the motrfile has me wondering if I should be re-evaluating the design of the highest-level wrappers...

Good night.

Coding 2022-07-17

2022-07-17T04:00:00-04:00

Okay, I did some work on writing today, and I did some more, but mostly I was working on getting an idea of how to write these docstrings. Here's where I ended up:

Need to have autodoc, so use sphinx.ext.autodoc
Sphinx-style docstrings are painful, so use sphinx.ext.napoleon.
I think I prefer numpy-style docstrings for most things...
But I don't want to double-annotate types, so I messed around and discovered that it's possible to mix docstring styles, to have google-style for Returns and Yields and numpy style for everything else.
I think autodoc_typehints = "signature" in conf.py is doing something.

Other things I found:

sphinx_autodoc_typehints and sphinx_toolbox.more_autodoc_typehints offer similar functionality (I believe one is a fork of the other), but I don't remember if I ran into a solid difference in my limited testing
I couldn't get TypeVars to consistently do what I wanted, but things might be different in a proper codebase.
These packages don't seem to put types on Yields fields.
Apparently none of this code supports Receives, which is documented in the numpy style guide, but doesn't seem to exist in autodoc. So, it doesn't look nice no matter what I do.
It is, however, possible to add custom fields, so I added these lines to napoleon_custom_sections in conf.py: ("Receives", "params_style"), and ("Type Parameters", "returns_style"). I figure the Type Parameters should go between Extended Summary and Parameters.
autodoc just generally doesn't seem to match up with how the styleguides think you should be documenting generators.

With all of this research, now there's just a little more I want to check before I try to document MOTR...

Starting in Python 3.9, I can do stuff like list[int]
I believe it should be possible for me to use int | str style syntax in annotations; I may look into bumping MOTR's minimum Python version after the next release.

Anyway, all of that gives me stuff to guide my attempts to actually properly document MOTR. The general idea I have there is to write documentation that's good enough for fixing the design issues I hit, then look into adding a pylint stage to the current motrfile, unless I can find a flake8 plugin for stuff like "documented names match the names in the code". Maybe do those in the other order, actually, so I have a checklist to work through.

Anyway, I'm going to take care of some things, and then do a bit more writing. I don't want the night to get completely away from me again.

Good night.

Diary 2022-07-16

2022-07-16T04:00:00-04:00

Well, things were kind of disorganized today, so I'm going to just take this entry easy and plan for tomorrow:

I need to read stuff before I can get back to the writing.
Before I document more about MOTR, I want to evaluate a few different options for documenting attrs-decorated classes, because I'm not sure what strikes the right balance in terms of generated documentation vs the raw file vs the runtime introspected data.

Anyway, I've got some completely unrelated writing that I want to do as soon as I can, so I'm going to wrap this up now and get back to that.

Good night.

Coding 2022-07-15

2022-07-15T04:00:00-04:00

Nothing dramatic on writing today, because I've got to do a little research.

As far as drama goes... I got to work on that bug in MOTR, and in solving it, uncovered a much bigger design issue. Let's see if I can explain it...

So, Parametric objects are wrappers around a function, and bring in associated metadata that allow the library to basically construct product iterators, but iterating over a variable twice is basically the same as iterating over it once. Now, these functions can technically produce anything, but they have affordances around producing the IO types that handle coordination between actions in the core. Basic things they try to support:

"Act like this variable takes on this value, for this function" I didn't run into this when I was hitting bugs, but there's some subtlety here
"Conceal iteration over a variable, unless the outer function also iterates over it" This is needed for things like "here are twelve outputs; they must all be used as inputs to the same action"
"Prevent iterating over any variable that this function doesn't know about" This is needed to prevent multiple actions from producing the same output

If you read that last one carefully, you might be able to figure out how my initial implementation can, and just now, did go wrong...

My initial implementation said "You can iterate over variables that this function doesn't know about." But, it's actually totally valid to iterate over "extra" variables, as long as they don't take on more than one value. Now, the fields involved are just obscure enough now that I think I should take this opportunity to document them...

provided_labels represents the variables that the wrapped function should vary with respect to
required_labels represents the variables that something should iterate over; either this function, or something else
handled_labels represents the variables that the function will gracefully handle iterating over
allowed_labels represents the variables that it is acceptable for something to iterate over
flexible is an auxiliary variable required by the user interface, and not terribly interesting for these purposes.

Relationships between these variables:

handled_labels and allowed_labels are both supersets of provided_labels and required_labels. One of the validators responsible for maintaining this is oddly-named, and I need to investigate it.
It's permissible to basically "copy" handled_labels into allowed_labels, to cut down on the acceptable options for iteration.
It's permissible to "move" everything from provided_labels into required_labels (roughly speaking), to create a function that "doesn't provide" the variables that it needs something to iterate over.
It's permissible to add information to flexible.
It's permissible to cut down on the provided_labels, but the logic involved is somewhat elaborate.

The basic things I need to handle are:

Instead of not allowing iteration over variables not in allowed_variables, I need to require that those labels take on only one value.
Something similar needs to happen with the inject_registry helper function. For each key it injects, it needs to make sure that either both values are length one when the key is used in iteration, or that both values are the same when the key is used in iteration.

It's not immediately obvious to me how to do this, because the documentation is so bad right now. I'm basically going to have to do some documentation and renaming passes in order to make sense of everything. One code change I'm considering, is to create a custom container type for parameterizable values, but I'm not sure if that's necessary. Documentation first.

Well, sleep first, documentation later.

Good night.

Diary 2022-07-14

2022-07-14T04:00:00-04:00

I got a bunch of stuff taken care of early-ish today, so I had plenty of time to write. I got to what seemed like a major milestone back when I was planning these edits, but functionally what it means is, I'll hopefully be done with the current pass in a few days, and then I can review everything and plan more editing passes.

Anyway, I'm done with that for tonight, let's see what else...

I'm going to try to work on MOTR a bit tomorrow, and after I fix the Flex-related bugs I identified, I'm going to try to put some code in core to improve the test assertions for the higher-level code. Basically, I want to be able to guarantee some properties of the dependency graph, and the code to calculate those properties should live with the code that defines the graph. I feel like those checks should be available to the main program logic somehow, but I'm not sure what the right interface to that is.

Anyway, I'm tired and I ought to wrap up.

Good night.

Diary 2022-07-13

2022-07-13T04:00:00-04:00

I ran into some trouble with writing, so I switched gears to some other writing that I can come back to later.

Besides that, I've been poking at some stuff related to MOTR, but not doing any of the big work there yet.

I guess I mostly just focused on the kinds of stuff that I don't really blog about, once again. I'm tired, so I'm going to try to get to bed a little earlier.

Good night.

Weekly Roundup 2022-07-12

2022-07-12T04:00:00-04:00

Wednesday: Was chill, though I did have to work through some technical issues.
Thursday: A little more technical stuff, and it seems to be basically settled at this point.
Friday: I worked on writing.
Saturday: I did some writing, and rambled about software metrics that are in this weird space of people at conferences going "Wow! Cool tooling!" and every paper I find on it managing to include a sentence to the effect of "Nobody knows what some of the words required to define this metric actually mean."
Sunday: I noted some work I'm going to have to do on MOTR.
Monday: I prioritized that work.

Next week, I'm going to try to make enough progress writing that I feel okay taking a break.

Coding 2022-07-11

2022-07-11T04:00:00-04:00

Fortunately, I made good progress on most things today. Unfortunately, coding stuff was not part of "most things". Still, I'm used to doing it like this on the weekends...

Instead of touching the code, I did mess around with software metrics stuff. The main conclusion I reached was that I should stop messing around with software metrics stuff.

Reviewing the notes I took yesterday, and making guesses about priority:

Review FlexOut and compare usage with LessFlexOut
Inline globals that are only used once
Unify creating the pip args label and the python version adaptor
Mark low-content flake8 wrapper code for removal, but do not touch it yet
Prototype a unification of creating Put objects and their associated Labels, but do not change the code yet; evaluate how the prototype would handle arguments for other wrappers

The first one is the highest priority, because it represents a bug arising from the... "design". Everything else is "how can we make it more convenient for library writers to deal with this code?"

Anyway, I should not hold off on getting to bed.

Good night.

Coding 2022-07-10

2022-07-10T04:00:00-04:00

I made some nice progress on writing, but I'm used to doing coding entries on the weekend, so let's talk about where I am with MOTR.

I wrote the first test of the pytest wrapper, which also represents a test of the python wrapper. Here's what's left to test, in approximate priority order:

pyinstrument (I looked into some alternatives, and they weren't satisfactory; I may end up filing some bugs later, once I've closed out some issues on my end)
coverage (including several output formats)
limit-coverage
Proper coverage of "build", which should be renamed to "pyproject_build", for both clarity and correctness

pyinstrument should be really straightforward, the rest, somewhat less so.

Now that I've done a few rounds of iteration on the existing wrappers, it might also make sense to revisit how I'm laying out these modules. I'll try to take some quick notes on that now...

The python versions adaptor and the pip arguments label always accompany each other; this didn't use to be true, but I had to do some refactoring that made it true.
Sometimes the Entry alias is used just a little, and sometimes it's used a little more.
The Program subclass does not always accompany the adaptor and label.
Some of these globals, like COMMAND, could be inlined without hurting anything.
The BASE global looks like it should be possible to write a helper for.
The python wrapper's base() function looks like a candidate for writing a helper function, but I'll need to compare it to the other runner wrappers' implementations, so I'll have to write them first.
The flake8 wrapper has some options that should be replaced with a variadic function of extra dependencies.
There are some issues with HTML_LABEL in the junit2html wrapper that indicate that I need to revisit the new LessFlexOut class.
There are a variety of paired parametric_command.Put objects and Labels, and I should take another shot at unifying them.

Okay, that's a bunch of work that I can't prioritize right now. Tomorrow, hopefully. For now, it is late and I should sleep.

Good night.

Diary 2022-07-09

2022-07-09T04:00:00-04:00

So, I got distracted from writing. With other writing. On the on hand, it's good practice. On the other hand, this is just doing a mix of not tackling my existing editing workload, and adding more for later, so that's a little obnoxious.

What else have I been thinking about... Well, the more I've thought about Halstead metrics, the more I think to myself "I'm not sure these can produce directly comparable metrics between languages, and maybe each language or language family should have metrics derived from a set of principles" because I'm genuinely not sure how you'd come up with a coherent way of classifying tokens that accounts for all of "pure functional language that doesn't allow you to bind anything to a name", "tape-based language that also doesn't allow you to bind anything to a name", and "stack-based language that allows for quoting; like, either quoting somehow turns an operator into an operand, or not quoting is an operator!"

I wouldn't think about this so much if I didn't watch people singing the praises of Radon for measuring Python code, and if Radon weren't so... questionable.

I'm going to try working through some of these on my own code, just for the heck of it. That's going to take a bit, so I'm going to publish now.

Good night.

Diary 2022-07-08

2022-07-08T04:00:00-04:00

I got to some rough bits in writing, so this was a bit of a slow day. Although, it's a little hard to judge, because I'm tracking net counts, and part of the editing I did was to delete old stuff that I don't need to refer to any more, so I probably made a little more progress than a naive count would indicate.

Anyway, maybe like another week or two of working on this, and then I can collate all of the TODOs I left for myself, and then basically do it all again. Yaaaaay...

It turns out I can't think of anything else to do in the next few minutes, so I should publish this and get ready for bed.

Good night.

Diary 2022-07-07

2022-07-07T04:00:00-04:00

Writing went well again.

After I messed around with Firefox's new download flow, I switched some stuff back how they wanted it, and some stuff how they didn't. At this point, I'm genuinely not sure if I'm actually using the new flow, but it says I am, and all of the overrides I have set up are supposed to be supported, so I think I'm prepared to not be nervous about getting the rug yanked out from under me.

I'm also pondering some software stuff, but I should really see about finishing up the current, months-long push on MOTR before I focus on working on the tooling.

Aside from all of that... I don't know, I should try to be a little more on top of the stuff that I do every day and don't really talk about in these entries? We'll see.

Anyway, time to wind down.

Good night.

Diary 2022-07-06

2022-07-06T04:00:00-04:00

I had a good day of writing; I probably won't be able to replicate that on days that I don't have off, but we'll see...

I updated Firefox, and that seems to have done some weird things to my extensions. Hopefully, that clears itself up later as more people use them. The other thing I'm expecting from this update is that they redid the downloads workflow in response to user feedback, so I'm giving that a chance. I just need to have something to download in order to test it. I'll figure that out later.

...

I poked around a bit, and stuff that I thought wasn't working before, period, now seems to be working, so, um, confused but happy shrug to all of that? I'll need to use it in anger to see if the download panel bothers me, but I think I've got the rest of the workflow the way I'd like it.

Anyway, I'm going to take care of some other things after I publish this post, and then try to take things easy.

Good night.

Weekly Roundup 2022-07-05

2022-07-05T04:00:00-04:00

Wednesday: I worked on MOTR and writing.
Thursday: I did some work on the writing, and figured out one of the bugs in MOTR.
Friday: I worked on stuff, sort of, but mostly took it easy.
Saturday: I reviewed some of the high-level wrappers in MOTR.
Sunday: I did some work on MOTR, and it was kind of a mess, but things came together.
Monday: I worked on writing, and a bit on MOTR.

Next week, I'm going to try to get over humps with the writing and with MOTR.

Coding 2022-07-04

2022-07-04T04:00:00-04:00

I was mostly focused on stuff besides code today, and I made good progress on those things, so that's good overall, but let's see what I can do right now.

In general, I should be working on writing helper code to inspect MOTR's core dependency graph. That's going to take some careful thought, though, so what's something I can work on quickly for now?

Well, the obvious solution is to update the python wrapper so it can handle the pytest wrapper. I think that basically, all I need to do there is update the BASE object to be the correct type.

All right, that was pretty straightforward. With a few tweaks as I go, that should have me ready to finish testing the wrappers.

For now, though, I just want to take a break before I get to bed.

Good night.

Coding 2022-07-03

2022-07-03T04:00:00-04:00

All right, I got the code to apparently work. I'm a little out of it as a result of all of the weird special cases I added to paper over the issues I was finding. I started documenting the experience here.

Basically, I added a bunch of extra wrapper classes to try to more accurately model the behavior I want out of the code.

Aside from the fact that I can't really describe any of the new code right now as "something I'd like to show off", the diff from today is so big that I'm not sure I can pick out any part of it as representative.

In any case, I'm going to publish this, consider some other things I wanted to work on today, and when I get back to it, start thinking about how to make the tests more thorough.

Good night.

Coding 2022-07-02

2022-07-02T04:00:00-04:00

I was poking at MOTR with new metrics stuff, and it had some recommendations, but making such changes isn't worth it until I've done a round of normal refactoring. I have a few other things I want to do today, so I'm going to try to get through this entry quickly.

My basic goal for now is to re-assess the pytest wrapper, and figure out what's going to need to change.

Comparing against other wrappers, I need, in approximate order:

Program subclass
Python versions ValueAdaptor
Pip args Label
Entry alias
CommandBuilder alias
OptionBuilder alias
COMMAND global
BASE global; things get tricky here, because as part of my paranoia about letting these programs touch the disk willy-nilly, I right now have it try to default to providing -p no:cacheprovider; on balance, it probably makes sense to ditch this and move that to the pytest config file
So, assume that NO_CACHE and DEFAULT get deleted, and BASE gets the same niceties as the other wrappers
The last thing in the wrapper, then, is the junit_xml function, which I now remember I never finished updating the code so that didn't need to be a function.

So, there are some fairly mechanical updates to the pytest wrapper, but before I deal with those, I need to:

Disable caching in the pytest config
Rework motr._api.helpers.reports.accumulator to get data from a parameter and also inject additional segments, forcing the file argument to be just a basename; this is technically less flexible, but I don't think it matters

Anyway, I'll try to get to those this weekend, and try to handle other things now.

Good night.

Diary 2022-07-01

2022-07-01T04:00:00-04:00

I was feeling kind of out of it today, so I didn't do any editing, outlining, or writing, and instead messed around with tightening things up in MOTR.

Just messing around a bunch with removing typing.Any, mostly in favor of object. I think this isn't always possible, but when it is, I might as well.

Aside from that, I did think a little about writing, but, eh.

I'm going to try to get serious about MOTR's test coverage tomorrow. Once that's done, I can start making some of the improvements I've had in mind.

Anyway, I don't want to prolong things too much, so I'm done with this entry for now.

Good night.

Diary 2022-06-30

2022-06-30T04:00:00-04:00

I did the outlining. I'm going to have to wait a bit to see if I missed anything, but the majority of it should be in a good enough raw form.

Earlier today, I investigated some long-standing issues with MOTR, but I didn't try to fix them yet. Basically... Cement and Trio both try to register a signal handler, and the consequences of Trio not registering its handler is nearly 700 lines worth of traceback on a ctrl-c, and if I stop Cement from registering its handler, then that means I need to add another except block to the main module. I haven't tried it yet, but this seems like a pretty straightforward tradeoff.

I'd like to wrap up for now, but it's nice to have made various bits of progress today.

(This post's summary is a reference that virtually nobody can be expected to get. Oh well.)

Good night.

Diary 2022-06-29

2022-06-29T04:00:00-04:00

I got a little caught up making obvious fixes to MOTR, and now it's at the point where I want to step back for a few days, then come back to it and see if it's ready for the pytest wrapper tests. Some of the code I ended up putting in there feels a little questionable. Like, hold on, why does mypy allow this? But whatever.

I did a small amount of rewriting today. I think what I want to do in the time I have left tonight, is to rough out the skeleton for this first part.

...

All right, good, got that together. Part of the next thing to do is work on another parallel outline that'll fit into gaps in this one. This makes sense, but I don't want to explain how. Anyway, I'm making progress, even if it's not reflected in the metrics I'm messing with recording. Give it a few days.

And with that, I'm going to wrap up so I can collapse in bed.

Good night.

Weekly Roundup 2022-06-28

2022-06-28T04:00:00-04:00

Wednesday: I was in pain.
Thursday: I continued to be in pain, but I tried to figure out how I'm going to move forward with my projects.
Friday: I started shifting my focus more towards MOTR.
Saturday: I took a break from MOTR, and did... stuff. Not particularly polished, but it "works".
Sunday: I managed to make a bit of progress on MOTR.
Monday: In continuing with that progress, I once again ended up wanting to rip out and replace some code.

Next week, I'm going to try to make good progress on this stuff, but we'll see how it all goes.

Coding 2022-06-27

2022-06-27T04:00:00-04:00

The high-level layer on top of MOTR is somehow in even worse of a mess than usual right now, because I was putting stuff together, and I realized that there should be a way to make things easier to use, but it's running into some friction.

Basically, I think I'm once again running into helper functions that are no longer being helpful. I'm not certain, but I think the way out here is to add some helper methods to the classes that some of this code was wrapping, and expose those classes directly...

The difficulty I'm going to need to resolve, though, is that, for a runner-type command, we start with the command, add the args, and go, but for non-runner-type commands, we start with the command, and the python version, so I guess I need a little helper class for that. I'm getting confused enough thinking about this that I'm going to take a break, and try the following when I get back to it:

Make a minimal helper class for these purposes.
Replace all uses of the current helper class with the new/existing classes.
Make a note of what's awkward.
Address that awkwardness if it's really obvious how, otherwise gather more data, by implementing more wrappers.

I let things get late so now I am once again cutting off the entry awkwardly.

Good night.

Coding 2022-06-26

2022-06-26T04:00:00-04:00

All right, I did some work on MOTR. The recursive methods are untangled and another one added. The main thing missing from ParametricCommand is the change-up of the type annotations.

Anyway, I'm a little out of it, so I'm going to cut things off and see if I can get more done.

Good night.

Coding 2022-06-25

2022-06-25T04:00:00-04:00

I did something else today, instead of working on MOTR. As exhibit A against myself, I present, the first cell of the newly created Jupyter notebook, crimes.ipynb:

def base_call(*args, **kwargs):
    return

SENTINEL = object()


class inner:
    def __init__(self, /, sup, default=SENTINEL):
        self.__super = sup
        self.__default = default

    def __getattribute__(self, name):
        try:
            return getattr(super().__getattribute__("_inner__super"), name)
        except AttributeError:
            default = super().__getattribute__("_inner__default")
            if default is SENTINEL:
                # Should inject name info
                return base_call
            return default

    def __get__(self, instance, owner):
        old_super = super().__getattribute__("_inner__super")
        new_super = old_super.__get__(instance, owner)
        if new_super is old_super:
            return self
        return inner(new_super)

class BetaMeta(type):
    def mro(cls):
        for base in cls.__bases__:
            assert base is object or isinstance(base, BetaMeta)
        base_mro = super().mro()
        return base_mro[1:-1] + base_mro[:1] + base_mro[-1:]

class Beta(metaclass=BetaMeta):
    pass

I'm not sure if this always works, but when I tried it with a basic example, it seemed to work. This code is really brittle. In particular, I kept on having trouble trying to find a way to not need to do inner(super()), which was pretty annoying. Various things not working that kind of seemed like they should have. Oh, well.

Anyway, the goal of all of that nonsense is to allow for BETA-style inheritance. Is that a good idea? Well, with all of that code, there's a way to find out. Once it's cleaned up and gets some proper invocations of the descriptor protocol.

Anyway, I don't want to ponder whether that's "something anybody should have attempted", or whatever, so I'm going to publish this and wind down extra-early.

Good night.

Diary 2022-06-24

2022-06-24T04:00:00-04:00

I'm still putting together editing plans based on the feedback I got. I hope I get to the point where I'm done planning stuff out for a bit. Then, take a break for at least a few days. Then start doing the rewrites. This is supposed to be fun, so I don't want to force myself into the meat of it.

Anyway, moving on from that for now, I took a look at MOTR's codebase, and it should be no problem to change the report path code. It means removing a usage of a helper function, but if the helper function isn't being helpful then... don't use it. Here's about what needs to happen:

Factor the recursive code in ParametricCommand into independent methods
Add a field for representing "segments" of the reports path
Add that field to the relevant ParametricCommand instances, or into the helpers
Write a recursive method to combine the field
(It may be possible to write some kind of decorator to abstract out the recursion but that is not a priority right now)
Inject the field into a label that is re-exported by the reports helper module
Rewrite the reports helper to retrieve the injected label, combine it with an optional argument, and use that as the base parametric for an InputAccumulator

That's some of the tricky stuff worked out.

Other things that occurred to me as I looked into this:

It should be possible to wrap stuff relating to junit into a helper function
... which has to be applied to a Parametric, so after ParametricCommand.build() is called
Because the version information propagates "outward" from modules, it should work to convert the module field into a union that either has a module or an adaptor; doing this removes a line of boilerplate from various parts of the API, and might simplify things even more than that suggests
It may be possible to put all of this into a callable object, such that it's reasonable for a motrfile to have no explicit control flow. (Users could then choose to add control flow that keys off of environment variables or something. I'm not going to bother trying to stop these hypothetical users.)

All in all, exciting times ahead for MOTR, which is good, since I'd like to feel comfortable focusing on any other python project at some point. Anyway, it's late, and I should wrap up.

Good night.

Diary 2022-06-23

2022-06-23T04:00:00-04:00

Ugh... Feedback... So hard to take... I'll have another draft together in, like a few months? Seriously? That's what it feels like...

Anyway, I also did some planning for MOTR, and I think there are a few places I need to refactor and then augment, and then I can work on some of the logic required to handle generating paths for reports. This whole... thing... might be an argument, at least as far as most people are concerned, against trying too hard to generalize something. Because this has been... a lot. My basic plan is to break down ParametricCommand._build() so its internal structure isn't as fragile to change. The problem with it right now is that the method is recursive, and the stuff around the recursion can't be easily rearranged, but this doesn't look obvious to me. Basically, the function does two things, that are both recursive. I'd rather split this into two recursive functions that do one thing, and then I can coordinate them in ParametricCommand.build(), and also add other recursive functions that are required for report path generation.

I also need to figure out how I want to assemble the commands that will take advantage of this functionality. I have something in mind that seems like it's powerful enough, but it's probably too complicated.

Anyway, I'm still out of it from all of that feedback, and also trying to deal with hurting myself over the weekend, so I'm going to call it here and try to chill out.

Good night.

Diary 2022-06-22

2022-06-22T04:00:00-04:00

Good news: I got some really helpful editing feedback on my writing. Bad news: I messed up my knees over the weekend and I can't figure out how to make them not hurt.

All right, let's see what else is on my mind. Now that I can look at MOTR and think "this part isn't working how I want it to", I can work on nailing down requirements. Sadly, part of this process will involve getting my knees to feel better so I can use my laptop for more than a few minutes at a time.

All right, I'm not thinking of more to write here by standing here; I'm just annoying myself. Time to call it.

Good night.

Weekly Roundup 2022-06-21

2022-06-21T04:00:00-04:00

Wednesday: I got sidetracked by the obnoxious behavior of the "enter" key in Jupyter Lab.
Thursday: I did some writing, and planned to work on MOTR over the weekend.
Friday: I figured out that the Jupyter stuff isn't worth looking into now.
Saturday: I poked at MOTR a little.
Sunday: Travel took a lot out of me, so I made one small improvement to MOTR.
Monday: I realized that the interface I'm building right now is kind of wrong, and tried to figure out how to fix it.

Next week, I've got some writing that's nearly ready to go into editing, so I'm going to try to work on that. I'm also going to look into changing the higher-level concepts in MOTR to see if I can come up with something cleaner.

Coding 2022-06-20

2022-06-20T04:00:00-04:00

All right, I tweaked the code some more today, and updated the prototype for the pytest wrapper. I think I have the wrong interface there for now, so I'm trying to figure out how to make it better.

Basically, when pytest runs and creates XML results, those results have to go somewhere, and currently that's being specified by passing a path explicitly. To make that work better... there's a concept I had in here; it was called something like "segments", and it might still be, I don't remember for sure... Ah-ha, it's the path attribute on the python_helpers.PythonCmd, which could pass them to the command.Metadata in some way that would allow it to inject it into argument...

One potentially confusing possibility is to rework the installer concept to be for any path generated along these lines...

Okay, this is a bit too much to consider right now. I'm going to need to take some time to consider how to make this work.

By the way, to clarify what's wrong with the current system: the ideal here is that the motrfile creates a single object that can be combined with other "runner" objects, and each combination should put its output in a different directory. Right now, the mechanism for changing the output directory is buried inside several layers of wrapped objects, so this doesn't work.

I want to just mess around and let this percolate in my head for a while, so I'm going take things easy for now.

Good night.

Coding 2022-06-19

2022-06-19T04:00:00-04:00

I was traveling today, for longer than I expected, and I'm a little messed up. So, I'm not up for doing much, and I got just one thing done: I inlined some private globals that were only used once. This makes the code a little less cluttered.

I have various improvements planned, and one of them is to create a helper class that will make it possible to inline more of these variables, and hopefully make the interfaces created with these building blocks a little more regular.

Anyway, I want to get ready for bed now.

Good night.

Coding 2022-06-18

2022-06-18T04:00:00-04:00

After I realized just how out-of-date a version of CodeMirror Jupyter Lab is using, and that there's an active effort to update it, I decided to hold on on writing code for it until they've ironed out the major version jump there.

I'm feeling a little tired right now, so I'm just going to see what I can get done or planned for MOTR.

Let's see what happens if I try to combine the python version stuff... It messed with things a little, but I think it's better this way. Better keep going...

Okay, I made those simplifications. The next logical steps are to rename some of the remaining globals to remove redundant information, and to inline some other globals that are only used once and not meant to be exported. It might also make sense to write __all__ lists for these modules for documentation.

I think the thing to do is write a bunch of my ideas down by hand, and work on them over the next few days if I have time. Let's see about that...

Okay, it's written down, and I want to wrap up for now.

Good night.

Diary 2022-06-17

2022-06-17T04:00:00-04:00

All right, I did a bit of writing and editing today, so there's that.

I am blanking on anything else to say right now.

Maybe I'd like to do a little more research into my Jupyter/CodeMirror complaints.

So, according to CodeMirror, I need a custom command to do what I want. So, that's "adding [a property] to the CodeMirror.commands object". Specifically, I need to change the newlineAndIndent command.

Before I get too much further, I need to figure out which version of CodeMirror Jupyter Lab is using. Looks like... ~5.61.0? So, I should see what the code is like at version 5.61.1, probably.

Anyway, in order to use that information, I need to have access to the relevant CodeMirror object, which seems a bit confusing to me, currently.

I think I need to write some TypeScript that exports a function; eventually in its own tiny module. Then, a Jupyter extension that wraps that function by... injecting it into the CodeMirror object associated with cells using the ipython mode? (For bonus points, figure out how to change the tokenLexer function to dedent on ... and raise as well (but perhaps not raise NotImplementedError))

I think I'm understanding this a bit better; the key insight that's bringing lots of things together is that Jupyter appears to be using a version of CodeMirror that's over a year old, which made my bug report, um, incredibly pointless? Oops. Sorry.

Anyway, I'll get back to all of this later; for now, I should get ready for bed.

Good night.

Diary 2022-06-16

2022-06-16T04:00:00-04:00

I caught up on a few other things today, and worked on some writing. The big roadblock right now is, I need to do some editing, but I don't want to do some editing. Like, why won't the text spontaneously rewrite itself into a form that makes more sense? Rude.

Anyway, I'm going to be traveling a bit this weekend, but it shouldn't be as crazy as the last bit of travel. We'll see how much I can work on MOTR like that. I think I'll want to focus on writing tests for the pytest wrappers. I'll see, as I work on those, whether I want to focus on cleaning up the code or covering more use cases. Like, once I have tests for the other three cases in the motrfile, that should give me coverage over most of the non-error paths, so I'll get some confidence in its behavior.

I think sometime, I'd like to look more into claims that additional test coverage isn't really helpful, because I feel like there must be some nuance missing. Because it just doesn't seem to match up for me. Is it because I try to be cautious and write tests before using the code when possible, so code issues manifest as test failures rather than debugging sessions? Maybe. Maybe it's something else. But regardless, either my subjective experience is an outlier or doesn't match up with a quantified measurement, or there's some reason that what I'm doing doesn't match up with the circumstances of the studies I've heard about that are supposed to show that increasing test coverage doesn't help.

Either way, I should get ready for bed.

Good night.

Diary 2022-06-15

2022-06-15T04:00:00-04:00

I don't recall that I did anything in particular today, besides trying to catch up on various tasks. I think I'm basically caught up at this point, so that feels good.

I did mess around with code a little, trying to document my untested Unlambda interpreter, and polishing MOTR's tests slightly. They could do with a bit more polish... And I may have gotten slightly sidetracked from writing the interpreter in Jupyter Lab, and encountering behavior that only makes sense to me if the top-most multi-line statement in a cell, if any, is "supposed to" be the last thing in that cell. I've written up a question that summarizes my current state of knowledge relevant to this. If anybody knows about "the right way" or "a way" to make CodeMirror behave differently from Jupyter Lab, I'd appreciate the help.

Okay, it's getting late, and there's a bit more I'd ideally like to get doe tonight, so I'm cutting this off here.

Good night.

Weekly Roundup 2022-06-14

2022-06-14T04:00:00-04:00

Wednesday: I was physically and emotionally tired.
Thursday: I figured out how to make the debate over 0.99999... worse.
Friday: I didn't have the bandwidth to write a long entry.
Saturday: I started getting distracted by Unlambda.
Sunday: I kept getting distracted by Unlambda, but I was able to plan a bit for MOTR.
Monday: I still got distracted by Unlambda, but I was able to execute some of the plans.

Next week, I've got some writing I really want to finish up. And, of course, I want to make progress on MOTR.

Coding 2022-06-13

2022-06-13T04:00:00-04:00

All right, I... got distracted.

Point is, I did get things together enough to start working on the report helpers for MOTR. I've written the code, so if that doesn't just outright break, I'll work on bringing it into the existing wrappers.

Let's see... Looks like I got flake8 updated... And mypy updated... And... junit2html.

Okay, good job, me, that cut out a few of the giant block of imports at the top of all of these modules. It is kind of a drag to start working on a new one of these, and need to bring in like a dozen other modules.

In any case, it's way too late right now, so I'm out.

Good night.

Coding 2022-06-12

2022-06-12T04:00:00-04:00

So, I slightly got caught up in updating Unlambda interpreter code. The biggest issues currently are:

Haven't ported the parser and main loop yet
The implementation of the S combinator looks wrong, but testing it against one of the old implementations, appears to work

Anyway, let's ignore that for now.

It's late, so I'm going to see how quickly I can put together something helpful for MOTR.

Currently, I have the label for the python versions, and the adaptor associated with it. Now, that can be consolidated into just the adaptor, and the convention is to pull the label off of it.

Similarly, the command object groups with the label for pip arguments. That consolidation should wait until I've written tests against the pytest wrapper, to avoid over-specializing.

For the reports stuff, that needs a new module under the helpers package. For now, that can also hold the common code for the functions that get used for adaptors.

Those should all be good to look into tomorrow.

Good night.

Diary 2022-06-11

2022-06-11T04:00:00-04:00

First opportunity at me-time: chilling out, which apparently involves trying to relearn how Unlambda works, and trying to hold the whole model in my head. Didn't go so well, but I had fun.

To try to get a handle on this stuff, I went searching around for previous attempts I'd made. I found a few, but I think I was in the middle of stuff with them, so they don't necessarily... make sense...

Anyway, I'd be interested in coming back to them later, but I don't know if I'll go for them tomorrow. I'm trying not to plan too far ahead, or very far ahead at all really. Nonetheless, I hope I manage to scrape together the focus to make some API improvements to MOTR.

Anyone, nothing like that for now.

Good night.

Diary 2022-06-10

2022-06-10T04:00:00-04:00

I'm still really tired from traveling, so it's hard for me to think of an idea for a post.

I'm generally getting stuff done, even though I'm tired, but I should probably stop pushing it. I'm going to get some sleep, and try to have some fun me-time over the weekend.

Good night.

Diary 2022-06-09

2022-06-09T04:00:00-04:00

Still super tired and out of it, but something interesting occurred to me earlier today.

As you may know, there are recurring arguments over whether 0.99999... is "really" equal to 1. I noticed an interesting perspective to take on this. In decimal notation, it's not just 1 that has this issue. There's 0.5 and 0.499999..., and 0.2 and 0.19999.... And it extends to any product of powers of one half and one fifth, and any integer.

But it's different, for example, in ternary. In ternary, besides 1 and 0.222222..., there's any power of 0.1 times an integer.

But it's different in balanced ternary. Suppose we use the digits -, 0, +. Now, the highest balanced ternary number with a 0 in the ones place is 0.++++++..., which corresponds to 0.11111..., which should be one half. Flip the digits and add one, and we get that there's another representation of one half: +.--------.... And one is only +. By switching around the notation, we've ended up with a completely non-overlapping set of problem numbers! This means that either the "problem" isn't real, or it's happening with literally every rational number, and it's simply obscured by the choice of notation.

If the "problem" isn't real, then we should understand the multiple representations to form equivalence classes. Which isn't so strange. Consider that 1/2 and 3/6 have no digits in common in those representations, yet they're normally treated as the same number.

In the alternative... There are infinitely many distinct numbers that are collapsed into a single representation by typical positional notation. To handle the difference between these distinct numbers, some more precise representation is required to track them. In other words, if you believe that 1 ≠ 0.99999..., then point notation is literally not good enough for you, and you should be looking for alternatives!

*mic drop*

Good night.

Diary 2022-06-08

2022-06-08T04:00:00-04:00

Ugh, wow. I'm way out of it now that the weekend is over. Like, all of that was necessary, but I am emotionally and physically drained right now.

I want to get back into working on things, but I also want to collapse, and I also think I should work on getting back up to speed with some things that I did a bit less of when I was traveling. I think the best way to handle that is to just accept that this will be a short entry, and work on those other things after I post this.

Make sure to let the important people in your life know that you love them.

Good night.

Weekly Roundup 2022-06-07

2022-06-07T04:00:00-04:00

Wednesday: I had a slightly rough day.
Thursday: I had a slightly annoying day, but I decided to focus on relaxing, because that's important.
Friday: I got a little work done on MOTR, and freaked out a bit from other things.
Saturday: I had a rough day, that's probably going to be comparably rough to things after I publish this entry, but oh well.
Sunday: I got more tests passing in MOTR, and tried to consider what I need to do to give it a good interface for third-party developers.
Monday: I made notes about common code in MOTR. The problem with this code is not just that it's repeated, but that it's not really obvious why all of it is there, which makes it hard to write.

Next week, I'm going to try to clean things up a bit in MOTR. And get back to various pieces of writing.

Coding 2022-06-06

2022-06-06T04:00:00-04:00

All right, let's see what kind of common functionality I can pick out of these four-to-six modules...

Common things include:

The Label object for representing pip args
Specifying which type of entry point for the command
Putting this information together into a Command object
The Label object for representing python versions to install a set of packages under
Turning that label into a ValueAdaptor
Subclassing _program.Program
Putting together the Command, a helper function, and allowed exit code data into a python_helpers.CommandBuilder
Designating the reports prefix
Designating a directory within a reports prefix
Labeling that directory
Creating an output file under the labeled path, using some basically-identical function implementations
Assembling everything into a Part object of some kind

The remaining bits of unique code are sooooort of hacks around inadequacies in the high-level design. It should be possible to streamline some of this code by creating helper functions to put together ValueAdaptor instances given a starting path.

It also kind of seems like it would make sense to bundle together the reducer function and initial value arguments to the reduce_parameterization helper. I've also got the urge to break up the parametric.py module into several smaller modules, and convert the nested function definitions into classes with __call__ methods, but I'm not quite sure there's a concrete objective that would be satisfied by doing that, so I'll hold off on that for now.

In any case, it's late, and I'm tired, so I'm just going to kind of... end this post now.

Good night.

Coding 2022-06-05

2022-06-05T04:00:00-04:00

Okay, I made a bit more progress on MOTR, partially by doing weird hacks to the code. I've now got a basic test working for the mypy workflow. At this point, I could either try to plug some coverage holes by copying this approach with the pytest workflow code I've written, or I could step back and try to figure out which bits of the code I've written should be their own functions, etc.

For the latter course of action, I've got five-ish modules to review and see if I can make modules like them more pleasant to write. Making those modules more pleasant is important because... Okay, so, imagining a hypothetical future where MOTR takes off, I've categorized developers who will do things with MOTR into 4 groups:

Core developers, aka just me for now.
Extension developers, aka just me for now.
Developers configuring MOTR to run against a repository, aka just me for now.
Developers running tests against a repository that uses MOTR, aka just me for now.

MOTR is right now good enough for the fourth category, and I'm getting there for the third one, but it could be better, for sure.

These five-ish modules are important to the first two categories because the third category needs high-level representations of automatable tasks to be able to write a concise and clear motrfile. (Sadly, judging by the tests I've written, I can't expect any improvements in clarity just yet.) Right now, there are two ways these modules can be improved:

Factor out common code to cut down on boilerplate that the second group has to write.
Write helper functions to cut down on the freeform nature of the current interface. Some of the test code is "obvious" conversions from input types, and there's no "static" analysis of whether the generated commands make sense. I think it should be possible to address these shortcomings together, but I should plan the details carefully.

For now, I'm going to take things easy, and pick back up with planning the boilerplate reduction first.

Good night.

Diary 2022-06-04

2022-06-04T04:00:00-04:00

Okay, I got through the toughest part of the weekend, I think. I don't think I have it in me to write anything interesting, so I'm going to just publish now and work on relaxing.

Good night.

Diary 2022-06-03

2022-06-03T04:00:00-04:00

Ugh. I did my best to have a nice, relaxing day today. I think I really did. But a few things were working against me.

Just now, I had a bunch of tech issues: The travel website was, I think, gratuitously confusing, and I had trouble getting stuff to print after I got through the website. That should be all set, but maybe we will find a [FUN] surprise tomorrow.

Anyway, that's that taken care, as best I can do.

I also did a bit more work on MOTR today. I haven't gotten the new tests passing, but it should be straightforward to finish up. Emphasis on should. Getting as far as I got today involved writing a ~twenty-line helper function and making code changes that I really hope were justified and principled.

Oh wow, I think I'm crashing from freaking out over that website. Stupid neurochemistry. Okay, yeah, feeling bad; I'd better wrap this up.

Good night.

Diary 2022-06-02

2022-06-02T04:00:00-04:00

Work went a little rough today. I figured some things out, and then... technical difficulties. Completely out of my hands. Really annoying.

Anyway, now I'm off, and I'm ready to relax a little, and get ready to travel.

For now, though, I'm going to revisit some of the writing I was doing earlier, because I found myself really enjoying it, and I want to keep that up.

At the same time, I should probably try to get plenty of sleep, so I'm going to see about wrapping stuff up soon.

Good night.

Diary 2022-06-01

2022-06-01T04:00:00-04:00

Today I had an okay day, until I suddenly had a very rough day, then I worked on getting it back to okay. Nothing very bad, just gross and painful.

Anyway, I poked at MOTR a little because I felt like it, and played some games. My plan for now is to post this, and get some other writing done.

I'd best actually do that instead of spacing out for ten more minutes.

Good night.

Weekly Roundup 2022-05-31

2022-05-31T04:00:00-04:00

Wednesday: I wasn't up for doing much, so I didn't do much.
Thursday: I picked up a random side project from a few years ago, in order to unwind.
Friday: I continued on with that.
Saturday: I started working on the code that's needed for the MOTR rework.
Sunday: I apparently finished the rework. We'll see how it shakes out within the next few weeks.
Monday: I did some minor cleanups to MOTR's code.

Next week, I've got some travel happening, so I'm going to keep on taking things easy. After next week, I hope I'll be able to ramp back up, but I'll have to take things as they come.

Coding 2022-05-30

2022-05-30T04:00:00-04:00

I'm feeling kind of bleh, so I'm going to go over what I did today as quickly as I can.

I didn't think of any major updates to make to the tests, so I just focused on updating the mypy tests. From that, I now have a todo list of functionality to implement to stop the test from failing immediately. I'll look into that at some point. I anticipate this week will be a little hard on me, so I'm going to try to avoid going, like "I'm going to put in this much work", or "I'll do it by this time".

Right now, I just want to feel better, and I think part of that is going to be getting off the laptop for a while.

Good night.

Coding 2022-05-29

2022-05-29T04:00:00-04:00

And, here we go. After a bunch of confused swearing and writing more weird helper functions, I've managed to seemingly update the code and the active tests to use the new logic I planned out. The proof of this will be in updating the other tests and seeing how they work, but I ended up realizing something as I was working on this:

These tests are nowhere near thorough enough. I was only getting failures because I was managing to trigger weird errors. I need to update these existing tests to actually make assertions about the shape of the output.

I just improved the tests a little, but I think they need some more work. I'll look into that tomorrow, and then see about updating the other tests to the new API. Hopefully, that will prove things out, given that the tests that need to be rewritten, were a major part of the motivation for the rewrite.

Anyway, it's late and I'm tired. Again.

Good night.

Coding 2022-05-28

2022-05-28T04:00:00-04:00

All right. I'm a little tired and out of it, but I got started updating MOTR.

I've got some of the infrastructure required to create the values that will be passed into the new build() method. The major thing I still need to add is something to create a bunch of PathWith[VirtualenvArgs]s from a bunch of VirtualenvArgs. I could get elaborate, but I think I'll do what sounds like the least effort to me:

Default the input to just the default interpreter.
If the default interpreter is passed, don't allow any other interpreters.
Copy some logic from elsewhere in the code to convert the path data in the interpreter info into a common prefix and different suffixes.

I'm trying to implement that right now, but I'm roughing out the code and it's kind of a mess. I'm going to publish this post and hack on the code a bit more, but it's not going to get much cleaner for now.

Good night.

Diary 2022-05-27

2022-05-27T04:00:00-04:00

The data entry is done for now. Just got to keep relaxing, and try to get some satisfying work done over the weekend.

For now, I can't seem to focus on much, so I'm going to just publish this.

Good night.

Diary 2022-05-26

2022-05-26T04:00:00-04:00

I've been trying to unwind more, and it seems that today, that meant picking up a data entry project I was working on, a few years ago, apparently. I don't have a concrete use case in mind for all of this data once it's entered. Sometimes you just want to enter some data, you know?

Anyway, I do want to come up with something to do that's, like, adjacent to the information I'm collating, but I'm not going to rush it.

For now, I'm just going to have to resign myself to recording some mildly interesting metadata and... there is no and. Just space.

Good night.

Diary 2022-05-25

2022-05-25T04:00:00-04:00

Hm...

On paper, I'm doing okay at taking things easy, but, like... I really want to work on code or whatever...

But it's too late at night as I write this. That would not be a good idea.

Best I can do for now is putter around cleaning up my browser tabs a bit more.

Okay, that's enough for tonight. Time to wrap things up and get to bed.

Good night.

Weekly Roundup 2022-05-24

2022-05-24T04:00:00-04:00

Wednesday: I did some planning for sci-fi writing.
Thursday: I was stressed out about the AC not working. Or, it sort of works, but in a way that's usually not useful.
Friday: I spontaneously wrote a few hundred words complaining about games that I end up playing a lot when I play them.
Saturday: ...
Sunday: I put in a little more work planning out MOTR changes.
Monday: I think I figured out all of the high-level aspects of the plan for MOTR.

Next week, I think I need to step back from pretty much everything for a little bit. I think I'm feeling burnt out some. I'll hopefully be up for working on MOTR, but I've got most of the really hard stuff done there; now it's just a matter of changing bits of code to how they "obviously" should be.

Coding 2022-05-23

2022-05-23T04:00:00-04:00

Well, I basically had "Objective: Survive" for today, and I did that, so, it's time to relax and wait for the air outside to cool off.

While I wait, let's see if I can take some quick notes on MOTR. Yesterday, I determined that ParametricCommand.build() should take a ValueAdaptor[PathWith[EnvironmentArgs]].

Now, the flip side of that is that the corresponding Label needs to be populated in the Registry. Right now, Registry population looks like yield from api.cli.build.build(package_data). And creating the information that would be stored in the label looks something like py = api.cli.flake8._installer.PathWith(pathlib.Path(), api.cli.flake8._pip.VirtualenvArgs()). Please ignore the encapsulation violations I stuck in there because I don't want to think about that part of the API yet. To get things up and running, I need a helper function that creates a vector of that kind of data, and to have an idiom for adding that data to the Registry.

Something that occurs to me, looking at this code, is that it might make sense to separate the current changes() implementation into a Callable[[Sequence[Parametric[None]], Requirements[Registry]], Requirements[None]], or something like that. Basically, indicating "It's not expected that instantiating Parametrics be interleaved with building the Registry." That kind of reorganization is only appropriate after I've written some more tests, and I want to fix up the tests that I have, first.

Anyway, I think the last piece of the puzzle is the implementation of the ValueAdaptor[PathWith[...]], and that should just be return path_with.path. So, I just need a function to bundle that up with a Label, and that should be the whole plan to get this working.

I can't work on it now, because the air is soup.

Good night.

Coding 2022-05-22

2022-05-22T04:00:00-04:00

Ugh, the temperature is spiking and the AC just sort-of works. Gosh, who could have predicted this? Besides me. A week ago.

Let me see what I can do in terms of nailing down the changes to MOTR.

So, ParametricCommand.build() must pass a dict in the form {Label(): ValueAdaptor()}, where... the Label is probably a ParameterLabel[PathWith[EnvironmentArgs]], and the ValueAdaptor is a ValueAdaptor[PathWith[EnvironmentArgs]]. I think. Now, that dict can be trivially derived from a ValueAdaptor[PathWith[EnvironmentArgs]]. And the ValueAdaptor[PathWith[EnvironmentArgs]] can be created by decorating a Callable[[PathWith[EnvironmentArgs]], PathStr] with @adaptive(LABEL) where LABEL is a ParameterLabel[PathWith[EnvironmentArgs]]. Because of the common logic, it probably makes sense to create a factory function that converts labels appropriately.

Now, at the other end of ParametricCommand.build(), I've got a Parametric[CommandBuilder[...]]. That needs to get properly converted, which can be done by mapping a basic parametric extraction over the relevant label and passing that into the requirements method.

I'm going to get some low-hanging fruit before I go too far.

Got it. Simple method on Parametric to pull out some redundant code.

Anyway, let's see... I think this needs some helper functions to create simple Parametric objects given Labels...

Given those helpers, I can map CommandBuilder.requirements() over the Parametrics, and lastly, I need to convert the resulting Parametric[Requirements[None]] to a Parametric[None], which should be another simple helper function.

There's still more to sketch out, but it's getting late, so I'm going to quickly put out the helper functions that I'm pretty sure I'll need.

Okay, that's done. The next areas to really focus on are:

Creating the ValueAdaptors that will be needed for the new interface.
Figuring out what I need to be adding to the Registry objects in the tests.
Actually updating the interface.

Hopefully, I'll be thinking a bit clearer tomorrow. We'll see.

Good night.

Diary 2022-05-21

2022-05-21T04:00:00-04:00

I am in no shape to write tonight. I need to take things easy.

Good night.

Diary 2022-05-20

2022-05-20T04:00:00-04:00

Well, I kind of spun my wheels a bit today. There's supposed to be an idea of a plan of a schedule for getting the AC properly set up, so that's as nice as can be, I guess.

I think I might try sketching out some of the stuff I stuck in the summary two days ago.

Relevant background.

Now, I read over that, and I think about Nova Drift, Slay the Spire, Solitairica, and Vampire Survivors.

Technically speaking, that post only half applies to Solitairica, because you don't unlock within-a-run upgrades in Solitairica, so it's "just" really grindy. Like, come on, I can clear normal mode with a basic deck pretty much at will, apparently. Just give me the rest of the dang game, please.

Also, the first half of the post doesn't describe Vampire Survivors, since it actually does have fixed maps, so good on it. Second half is super applicable, though.

I'm sort of thinking of this now because my save files for Nova Drift (which I was playing a few months ago) and Solitairica, which I just picked up recently, didn't survive the laptop migration, so I ground things up in Nova Drift, which is a little annoying, and I ground a little in Solitairica, and that's giving me this weird double sensation of "Oh, the flow!" and "Please just let me have all characters, six slots, and all power cards, this is busywork".

Having established that I'm complaining, I'll now think about why these games might be like this. I think the draw of these games lies in part in the combination of "bite-sized play times" with "long-term progression". So, you're not signed up to anything that requires long-term investment or understanding, and you can just knock out a play session, potentially in an afternoon or so, but you're still making progress every time you do.

So, assuming I'm right about this being the reason for stapling together RPG progression and roguelite runs, is there a way to do this that I'd prefer?

Now, "not having the whole game" bothers me from a gameplay perspective, but there's kind of a symbolic issue here: When we upgrade things between runs, what are we playing as? We get some experience points or make some purchases, and somehow this... changes the options available to the brand new iteration of the character in the next run? Like, I'm making persistent progress, but I also have to re-buy all the potions and such that I want each time. Something that occurred to me as a flavoring possibility for these "per-run" upgrades is tactical modifications to the area the run takes place in. Like, you can set up traps and trip-wires or whatever, and when you move on to the next area, you don't get back the stuff that you deployed, because you used it.

Let's see how this idea interacts with the points in the linked post.

First off, I'm assuming that this is presented as a vaguely RPG-ish with very forgiving death mechanics by roguelike standards, in order to maintain the "bitesized run" concept. You have save files, each save file has a series of cleared zones, you advance by clearing a zone, which takes no more than a few hours, and if you die, it punts you back to the beginning of the zone.

"B-but permadeath!" "There's no way for dying to gum up your meta-progression in these games, so something's clearly off-kilter anyway."

Each zone starts with a shop that has a random inventory of tools for helping you clear the zone. Clearing the zone in certain ways can give you bonuses of some kind.

So, you've got a save file, the character is advancing in ways gated by the random shop inventory, I don't have strong thoughts about the zones, but it's a better decision to design a bunch of "levels that work" than to try to work out the principles for good level design from nothing and program them into the game, (if I want proc-gen for levels later, it will help to have examples), and the part about optimization vs crazy runs has me thinking of something else...

Suppose the kinds of items available from the shops were tied to some kind of "chaos meter" like from Mythic, where low chaos made it easier to obtain lower-tier and reliable items, while high chaos made available items "swingier" in various ways. Then, player conduct could influence the chaos meter up and down, to try and tailor the overall experience to match the play style. I don't know if this is a good idea, but it's an idea, to be sure.

Anyway, I'm really tired, so I'm wrapping things up for now.

Good night.

Diary 2022-05-19

2022-05-19T04:00:00-04:00

Uuuuugh. Didn't really manage to focus on anything today. I wrote a little.

I think I was in a funk because I'm having trouble getting building management to acknowledge that, hey, they should probably schedule that yearly maintenance that the AC here seems to need, before the weather turns completely intolerable this weekend.

Call me pedantic if you must, but I don't think the AC should be blowing air above room temperature.

Watch this space to see how angry I am in the third entry after this one.

Anyway, I let things get late, and the only thing to do now, really, is to get to bed.

Good night.

Diary 2022-05-18

2022-05-18T04:00:00-04:00

Gaaaaames... And writing that I won't describe here.

"Let's see if I can spin up something in fifteen minutes or so." is the first thing that occurred to me writing that sentence, and also a terrible idea.

For now, let's just quick review the writing I'm trying to focus on. I've got concepts for nine characters currently, but they're not very well-defined beyond skillsets. I think for each character I'm going to want to work out:

Background, including connections to off-screen characters
Appearance
Associated genre, including stories to read for reference.

Once that's filled in a bit, I think I should work on outlining.

For now, though, I'm very tired and I should get ready for bed.

Good night.

Weekly Roundup 2022-05-17

2022-05-17T04:00:00-04:00

Wednesday: I worked on MOTR because I wanted to unwind some.
Thursday: I made a small amount of progress on writing.
Friday: VIDYAGAMES and a headache.
Saturday: I did some planning for working on MOTR, and had a feeling I was missing something. This turned out to be foreshadowing.
Sunday: I wasn't up for doing much work before I published the post, but I got a bit done afterwards...
Monday: And, thinking it over, realized that the code needed another minor rewrite, which I started sketching out a plan for.

Next week, I want to take some notes about that rewrite, and get more prose writing done.

Coding 2022-05-16

2022-05-16T04:00:00-04:00

Okay, after the last entry, I drafted out a test using the not-yet-written modules that will be needed. Some of it will need to be updated to match the interfaces, but first, there's something pretty serious that I need to work on.

Recently-ish, I was having trouble accounting for the different Python versions that will be used by the various commands. At the time, I punted on them, by pulling them out of some of the higher-level data structures, and making them a parameter to the map method.

As has been kind of a recurring thing, I'd forgotten part of the design goals. The choice of interpreter is supposed to be recapitulated by the FlexIn object, which means it has to be stored under a ParametricLabel[PathWith[VirtualenvArgs]] in the Registry. I didn't work on this earlier today, but let's see if I can put together a quick sketch of what this would entail, in the next fifteen minutes or so.

Right now, the data is injected like so: lambda command: command.requirements(python). After the changes, that site is going to look like lambda command: command.requirements() (or potentially just passing the unbound method). An additional line will need to be added to get the relevant data into the registry.

So, let's have a look at the requirements() method...

Hm. I'm not seeing a way to dramatically change this part of the code. I think I need to look into the ParametricCommand class. That's where I thought this stuff was supposed to live, I think. That's the level of code that handles injecting information into the Flex objects, so let's see...

One possible way in is a pre-processing step in ParametricCommand.build(). Let's see where that's called from...

python_helpers.CommandBuilder.parametric() From an API perspective, this looks like an excellent place to add an argument for this purpose. Now, it's just a question of figuring out what that argument looks like...

So, here's what I've got:

ParametricCommand.build() gets some kind of new argument, and takes on responsibilities to code that argument so ParametricCommand._build() can use it, and process the result of ParametricCommand._build() so that the outer function now directly handles the requirements() call.
Parametric.map() gets trimmed down and renamed.
python_helpers.CommandBuilder.parametric() gets a new argument.

I'm not done sketching this out, but I think I'm done for the night. I want to try and cool off.

Good night.

Coding 2022-05-15

2022-05-15T04:00:00-04:00

I'm kind of stuck taking things easy until the AC gets set up or the heat breaks. The heat's going to kind of do that over some of the next few days, so I should be good as long as the AC is working before next Saturday.

For now, I've done some of the preparatory re-organization for the Mypy testing. I think I'd like to post now, and see if I'm up for more work after.

Good night.

Coding 2022-05-14

2022-05-14T04:00:00-04:00

Okay, I did a little work on writing, but not much. I need the air to be less harsh before I can really focus.

For now, I want to make plans for testing the Mypy wrapper.

So, Mypy is the next logical step, because it has xUnit output, so there are additional post-processing steps. The things that need to be linked up are:

Run mypy, create coverage report and xUnit file
Merge all xUnit files
Convert merged file to HTML

Parts of the command line wrapper should include:

The base command
HTML report output option
XML report output option
Source directory (should be same logic as flake8)

Looking over the relevant code, I wonder if the definition of FlexOut should be tweaked so it creates its own labels without them being passed in. That seems reasonable to me.

Then, we get into the combination. This needs:

The base command
A function to convert from a FlexIn to an Arg
Output option

Then, converting to html:

The base command
Input option (should take an input path argument)
Output option

It'll be interesting to see what weird oversights are lurking in the code, because I think most of this should be straightforward, and that makes me feel like I'm missing something.

Anyway, I'm not awake for much longer, so I'll wrap this here.

Good night.

Diary 2022-05-13

2022-05-13T04:00:00-04:00

Any plans to do anything with my projects today were sacrificed on the twin altars of "My head kind of hurts" and "Just one more game".

So, um, this is late, and I'm pushing it out as fast as I can. Oh well.

Good night.

Diary 2022-05-12

2022-05-12T04:00:00-04:00

I ended up doing other stuff and vegging out today, but I did have time to name all of the characters. After using the technique of "generate random words and derive puns or references from them" to make names, a whole bunch, I wonder if these names need some tweaking. But for now, I'm going to forge ahead.

And by "forge ahead now" I mean "Try to work on sketching out these characters in more detail, tomorrow", because I am really tired and I want to lie down.

Good night.

Coding 2022-05-11

2022-05-11T04:00:00-04:00

I had a rough couple of days, so I finished the basics of the planned renames to blow off steam. This was a little bit of a double-edged sword, because it turns out getting the renames done was a lot of typing, and I was trying to do it fast, so my hands feel mildly weird after all of that.

Anyway, I'm going to let that sit and come back to it on the weekend. I think I'll work on writing tests, unless something else comes up.

Anyway, thinking about writing. I'm not in the best shape right now for ~*~creativity~*~, but I can put together some plans, some stuff to fill in.

For the setting I'm focusing on, one way to sum it up is that a small group of people with very different fields of expertise are gearing up to fight off an alien invasion. So, I want to come up with a few different areas of expertise. So far I've got:

"The leader", who focuses on administration and recruitment, and probably won't be a viewpoint character.
"The investigator", who is there to solve mysteries. There will be some opportunity for normal crimes mysteries, but I want the focus to be on stuff like "what could have been the purpose of this weird artifact?"
"The technologist", who does general-purpose repair and development. The focus there is on "how can we use this new alien thing, or fix this weird problem that showed up in the alien thing that we're already using?"

Here are some roles that, on reflection, seem like kind of good ideas:

Farmer/general life support
Military experience seems really really important in the context of fighting a war
Theft/infiltration, maybe?
Propaganda/persuasion. This will come up with the direction I want to take things.

I think I'll call things there for now, and try and flesh this out and see what else I feel like is missing later.

For now, I want to unwind properly.

Good night.

Weekly Roundup 2022-05-10

2022-05-10T04:00:00-04:00

Wednesday: I made some progress on writing stuff.
Thursday: I thought about where I want to take this writing, and spitballed some ideas for conlang tooling.
Friday: I put together some ideas for representing glosses for conlanging, in a form that's easier to edit than writing a giant table manually.
Saturday: I started to draft out the grammar, using those ideas.
Sunday: I came up with new and hopefully better names for some, but not all, of the higher-level API concepts in MOTR.
Monday: I started on the renames.

Next week, I'll try to get more stuff done with writing. I don't know how much I'll be up for, because something got me painfully keyed up today, and I don't know how much I'll be capable of if that keeps up.

Coding 2022-05-09

2022-05-09T04:00:00-04:00

And, traveled again. I don't know if I'll manage all of the changes that I came up with, but I'd like to get started.

So...

I handled the first two I wrote up, but I know that updating Dynamic to Parametric is going to be a really big change. I think it's better to cut things short for now, and come back to this later when I have more time.

Good night.

Coding 2022-05-08

2022-05-08T04:00:00-04:00

Okay, traveled today, but I still want to try and do the work I had planned for MOTR, which was to...

Oh no.

It was to evaluate all of the ill-fitting names in the upper layers of the API and try to come up with better alternatives.

All right, let's work through these modules alphabetically...

arguments.py: Rename to command_builder.py, see how it treats me from there. Could stick Builder on the end?
command.py: Oof, not really sure. Also, this module contains one of the *Meta classes, which really need something nicer. I mean, maybe just renaming all of those classes to Metadata would help things?
dynamic.py: Ooooof. Another *Meta, that one should just get whatever treatment the others get. The bigger puzzle is Dynamic itself. Like, Dynamic is a pretty questionable name for a concept in this context. What is it? Well, it combines a bunch of metadata with a function that has a particular signature; and this lets it... Um... ThisIsUsefulISwear. Okay, stepping back for a sec. The point is to call the function with every valid combination of parameters. So the various names involved should say something about parameters. Let's try and consider Dynamic, Objects, and Items together, and oof. Now, whatever Objects is, it's agnostic to all of this parameterization stuff, so it might deserve to get the Registry name back. Items can be ParameterRegistry if I don't think of something better. (And ItemLabel turns into ParameterLabel.) And Dynamic can change to Parametric, or something better. The helper methods should mention parametric. inject_into_parametric, map_1_over_parametric, map_2_over_parametric, NoParameters, no_parameters, reduce_parameterization...
I can't tell whether the flex.py names are good or not, so let's put a pin in those...
Something feels off with the input_accumulator.py stuff (like that the core type is accumulating specifically into a _cmd.PathStr, and using adaptors to make types of other values work with that, but I'm again not sure what to do differently.
installer.py feels like maybe I want to divide it up. Before I changed the code to make the new tests... work... this module had much fewer classes in it.
invocation.py is confounding because somehow I've written another command builder, and it doesn't replace the others, it just builds on them, with an interface that is "nicer" in some sense that I'm not sure how to quantify. I guess the point is that this bridges the command stuff and the dynamic/parametric stuff, like a kind of... ParametricCommand. That's definitely clearer, in that you can look at it and mentally break it down into the questions of "what does 'parametric' mean?" and "what is a 'command'?".
items.py: the "Item-to-Parameter" change should handle all of this.
objects.py: the "Objects-to-Registry" change should handle it.

Also, for those last two, and maybe some others, I want to roll the "use TypedMapping" change out so I don't have to worry about code drifting out of synch.

So, I'll think this over in the morning or next week, and see what I think. For now, I really want to wrap things up.

Good night.

Diary 2022-05-07

2022-05-07T04:00:00-04:00

I didn't get as much done as I'd like today, but I translated some of my notes on conlanging into the format I was roughing out yesterday. I think I found some issues with one of the sentences, so I'll need to come back to it. Probably the course of action that makes the most sense is to consolidate all of my notes into a single repo, making sure to note anything that needs to be reworked. Before I copy in more glosses, I need to review the things I can do with custom directives, and see if I want to change anything about the processing.

Anyway, I didn't get much done today because I'm way out of it, so I'm not going to try to drag this out any further.

Good night.

Coding 2022-05-06

2022-05-06T04:00:00-04:00

For the stuff in the last coding entry, I worked out some stuff with Hatchling. I'm really not confident I'm planning to use it as intended, but, like, whatever.

For this entry, I'm not touching that, or MOTR. Instead, I'm doing a little sketching of stuff for conlanging. Basically, I'm trying to figure out the right way to support glosses.

To start with, I'm thinking about Sphinx, and figuring I'll go for something like this:

:word:`gloss()-SG-gloss(stem)`

.. gloss:: Free translation.

    - `gloss(stem)`
    - `gloss(stem)-AFF`
    - etc

Where the :word: role has a parser to convert the given minilanguage into helpful Python objects that then get rendered in various ways, and the .. gloss:: directive performs further validation. Specifically, that either all of the roots are filled in, or none of them are.

With these ideas and simple dummy implementations, I should be able to get started writing up glosses under source control, in a format that doesn't involve overmuch tedious fiddling with control characters, or manually aligning things.

Anyway, I'm still feeling iffy (it might be the pollen) so I'm going to get to bed now.

Good night.

Diary 2022-05-05

2022-05-05T04:00:00-04:00

I wasn't terribly focused today. There's something weird and gross in the air; it might just be heat and humidity.

I took a few more notes for writing. I'm fired up about this concept now that I want to do some kind of like... goal-directed outlining. My mental image of how it would be presented is, like, a series of short stories that could, conceptually, be grouped into volumes. So, I should be able to iterate on process in a somewhat straightforward fashion, once I have the focus to really get into this.

I also tried to sort things out when it comes to conlanging, and it kind of feels like I've somehow circled around in terms of tooling. Like "It sure would be nice to have a table format that works well with version control, so I guess that needs to be some kind of text, but I know I hated laying out these tables in ReStructuredText, so I guess I want to represent them in Python somehow, so I guess I do need a representation of at least the glosses..."

I'm fine putting that together at some point, but I want some idea of how I'll get the data back out. I'm sort of wondering if there's some way to pull data from a Jupyter notebook into Sphinx...

Anyway, I'm done for tonight. I can't be up much longer.

Good night.

Diary 2022-05-04

2022-05-04T04:00:00-04:00

Let's see about doing some quick writing. I've got these sci-fi settings I'm interested in. Let's see about fleshing out my notes a bit more.

...

Dang, that was quick but effective in moving things in a direction that interests me.

If the techniques I'm messing with can get me results this quickly, I think that means I should set myself some goals for "what needs to be decided before I start drafting", and then get to it.

Anyway, I'm going to wrap up for now.

Good night.

Weekly Roundup 2022-05-03

2022-05-03T04:00:00-04:00

Wednesday: I had more problems than usual with technology.
Thursday: I tinkered with code that could potentially help with some stuff I want to try out.
Friday: I messed around with other code, then worked on story concepts and got those somewhere interesting.
Saturday: I, um, relaxed.
Sunday: I cleaned up MOTR's test code a little, and took some notes on how to improve the interfaces.
Monday: I acted on those notes, and decided on the next thing that needs work: the dang names.

Next week, I'm going to try to focus on writing more.

Also, I was messing with mypycify earlier today, and I'm still having trouble finding a sweetspot of "easy to configure", "allows using mypyc", and "can turn mypyc off to create a pure python wheel". I'll try not to focus on that too hard, because I'll probably be most interested in it once MOTR is in better shape.