Three Dollar Quill

Well, there's another day of vegging out, and then doing some quick incremental work. The big thing I realized was that, with a hopefully more sensible idea of what kinds of iterators I want to work with, I've got a larger testable surface than I did before. I'm right now gradually building up the iterator combinators that will take things from "a file handle" to "a stream of words".

The general shape of things I'm aiming for is, different types of input get coerced to a stream of "plain-text characters", which then get all whitespace converted to space characters. At that point, I can apply whatever chunking logic I want to the stream. For now, I'm only going to support basic word-based chunking, which should be good enough, and focus on supporting additional input formats, like epub.

Let's see, what should be a priority? Plaintext, HTML, epub, PDF, odt, docx, probably something else I'm not thinking of currently...

Anyway, it's late and I should get ready for bed.

Good night.