Three Dollar Quill

What I know about pip's new resolver:

• People wrote stuff about how pip isn't a proper package manager, and I think the new resolver is supposed to address that.
• Supposed to make it harder to end up in an inconsistent state. In my own code, I mostly work with small virtualenvs, but at work, we addressed our various packaging woes by pinning every single package version, which sort of worked.
• Everything I've read about the new resolver makes it sound like it's a pretty pervasive rewrite.
• In particular, some functionality that was officially never supported was removed.
• That last sentence is salty because I got the idea for using said functionality from the documentation introducing one component of it; that documentation has been unchanged since the feature was introduced, so I'd sure as heck like to know what the original intention was supposed to be.

Okay, here's the deal.

• I'm developing groups of related packages in concert, and for testing and packaging purposes, I need to be able to install these packages, which depend on each other, through pip (or some other PEP 517 frontend, which I don't think really exists yet).
• I want to install my copy of a package, even if it's also available on PyPI, and I don't want to have to think about whether it is.
• I've attempted various ways to accomplish this, but the easiest way I found to do it with the old resolver is to dynamically define constraints files that use absolute file URLs to point to the project paths. (I could potentially make this faster by building wheels, but I'm not feeling bottlenecked by pip currently.) (I also added normal constraints on the third-party packages so I don't have to duplicate those in all of the pyproject files, but those constraints could live in a separate static file.)
• Currently, this doesn't work with the new resolver, because constraints "were never supposed to work like that".
• But there's enough demand for the feature that there's a design sketched out, and it'll happen when...
• When they get a PR.

Now, if I try to write that PR, that's a lot of effort on my part, and it could be pretty complicated. But consider: my alternative is something like devpi, which I would have to commit to setting up on every machine I do personal projects on. If I can manage to implement this (or help someone else do it; I just want it done), then I don't have to change my workflow, and it'll continue to just work, with only the bare minimum required setup. (Like, this current project's code is just... not going to work without Python 3.8 or later.)

So, what do I think I know about what has to change?

Well, old-style constraints are basically old-style requirements in a funny hat, and new-style constraints are their own specific thing. I believe that getting URL constraints to work requires the creation of a new kind of constraint, so the first thing I want to look at is how new-style constraints are defined, and used.

So, let's get into it.

Okay, some of the logic lives in pip itself, and some of it lives in resolvelib. Or is resolvelib vendored? Not sure the modules match up. Okay, looks like any changes I make need to be in resolvelib (or packaging? Looks more likely to be packaging) first, then the vendored copies need to be updated.

Okay, I confused myself several times looking into that. Here's what I've got currently:

• Some of the logic that is relevant to this lives outside the vendored packages, so doing this will involve some non-trivial changes to pip's code.
• I don't know how flexible the vendored packages are, so they might need some changes.

To get started, I decided to just follow along with the development documentation. I've got pip cloned, and tox is running tests. There are a few failures so far; I'll have to see later if those are expected, or something I need to fix in my environment. It's also probably going to yell at me about missing Python versions, since I didn't try to reach parity with what pip supports.

I don't know how much more I'm going to get done tonight; I want to just see the baseline state of the fresh clone. I should probably also mention this in the GitHub issue, make sure to communicate, like, basic things.

Update from running tox: it is, in fact, complaining to me about missing pythons. Including some that I see installed under pyenv. Oh dear, this could get messy.

Well, I'll work things out tomorrow. For now, I'd like to wrap up.

Good night.