Three Dollar Quill

It's a little earlier in the day, and I'm having trouble concentrating on what I want to concentrate on, so I'm going to try to lay out what MOTR does and see if I can find a better way, or if it actually does just all work.

(By the way, some of this stuff is slated to be potentially renamed. I'm going to go with the names currently in the code for now.)

So, let's start near the end. The end goal of running the motrfile is the creation of an object of type motr.core.registry.Registry. The registry contains several mappings:

• From motr.core.runner.RuntimeAction to motr.core.registry._ActionData, which tracks a set of motr.core.target.Target objects that have to be satisfied according to the parameters of the registry, before the key action can be executed.
• From motr.core.target.Target to motr.core.runner.RuntimeAction. This indicates the action that must be executed to satisfy the target.
• From motr.core.target_name.TargetName to a set of motr.core.target.Target. Every target name is user-selectable at execution time using the -t argument.
• And a set of motr.core.target_name.TargetName objects that should not be selected by default.

These fields are not modified directly. Instead, the class defines a require() method that takes an argument that is an instance of one of the types in the Requirement union:

• Action adds a runtime action to the registry.
• ActionInput adds a precondition to an action.
• ActionOutput adds a result to an action.
• TargetName associates a name to a target.
• SkippedName marks a name as not selected by default.

There are some undocumented requirements about ordering in order to avoid cycles, but one of the really important parts of this is that it should be the case that, if new_registry == old_registry.require(requirement) (without any error), then new_registry == new_registry.require(requirement). This allows me to play kind of fast and loose with requirements at the higher layers.

The next type that matters is motr._api.requirements.requirements.Requirements, which is just an alias for typing.Generator[motr.core.registry.Registry, None, T_co], where T_co is just typing.TypeVar("T_co", covariant=True). This type is used as a return type all over the high levels of code.

The motr._api.requirements package also has some modules that provide helpful wrappers around the Requirement classes, in the form of tiny helper functions that coordinate the creation of instances of those classes. I'm going to skip over them unless I realize that they're somehow important.

The next level up is the modules under motr._api.actions. One of these modules, io, defines more of the types used all over the place in the higher levels. Basically, wrappers around generic values, but marking them to be either an input or output of their associated action. The other modules are:

• cmd, which defines a Cmd class that is a runtime action to execute a command, and a cmd_ function that handles constructing a Cmd instance and putting out all of its associated requirements.
• mkdir, which defines a Mkdir class that is a runtime action to create a directory, a mkdir() function to wrap constructing a Mkdir instance and putting out the associated requirements, and a make_parent() function that does the same, but with the parent of whichever path it is passed.
• write_bytes, which defines a WriteBytes class that is a runtime action to write a sequence of bytes to a path. The bytes are constant with respect to the registry. There's also a write_bytes() function that wraps constructing a WriteBytes instance and putting out the associated requirements.

Then, there's the motr._api.build module, which just defines a function to convert a Requirements object into a Registry.

I believe this is the point after which I ended up building a whole lot of stuff that isn't actually used yet.

I don't remember what order any of this is in, let's see...

• cli/*.py
• cli_types/*.py
• package.py

The next place to look is into cli_types.

The easy ones:

• entry and program: define marker types for coordinating the combination of command line snippets.
• label: defines a type that can be used as the key to a mapping, in a way that provides type information about the value to Mypy, depending on the mapping.- objects and items: mappings that use Labels as keys. objects.Objects maps Label[T] to T, and items.Items maps Label[PVector[T]] to T. items also defines some helper functions.
• installer defines a protocol. Classes that implement this protocol provide a means of combining with other objects of the same class, and of converting a command name to an absolute path to that command, in a dynamically created environment based on the data within the class.

Then things start getting a little more elaborate. I'm gong to try to focus on the stuff that probably won't need to change, if I do need to change something.

• dynamic defines the Dynamic[T_co] class, which is a wrapper around a typing.Callable[[Objects, Items], Requirements[T_co]]. It also defines a bunch of helper functions and classes. Basically, what all of this is for is to have facilities for combining Dynmic objects, with the end goal being that it matches up with the input type to a function that I haven't mentioned here yet.

Here are the remaining modules:

• command defines a Command class that bundles a command name and the label of an installer, to indicate "this command, in the associated environment". It also defines a CmdMeta class, that represents a fragment of a command-line invocation, along with the metadata required to properly call the cmd_ helper. IMPORTANT NOTE: This module does not handle module-type commands properly.
• arguments defines Command and Option classes, which are wrappers around CmdMeta objects. The Command class also adds more of the metadata required by cmd_.
• input_accumulator defines a pair of related classes: InputAccumulator and ValueAdaptor. These classes work together to build up a Dynamic[Input[motr._api.actions.cmd.PathStr]], where PathStr is a helper alias.
• flex defines a protocol and classes that implement it. The protocol is concerned with transforming some kind of Dynamic into "the right" form for requiring or providing data. IMPORTANT NOTE: The FlexOut class seems like the likeliest candidate for implementing the behavior I was looking for yesterday. There may need to be other changes to fully support it; I'll go into my reasoning below.
• invocation defines the Invocation class and a bunch of helper classes. The Invocation class is basically the dynamic version of the arguments.Command class. It's meant to be the, like, ultimate form of all of this in terms of fully generic code.

So, what I'm basically looking for to handle the changes is to have a single Invocation object that can have something passed to its invoke() method that results in regular changes to the paths and names of some of the outputs it defines. I think that looks like...

invoke takes an Objects argument that gets somehow injected into the Dynamic[_arguments.Command] that it returns. (Another possibility is that Invocation objects have an Objects mapping on them...) Meanwhile, FlexOut should be somehow incorporating the values from the injected Objects into the generated paths and the output names.

FlexOut makes sense as a target than the slightly-lower-level modules, because the lower-level modules don't use Dynamics, and those are the most straightforward candidates for changing the data around like this.

Anyway, I'm going to need some time to consider some of this, so I'm going to wrap up for now, and try to deal with the clocks changing.

Good night.