17:58:05 <Emperor> #startmeeting
17:58:05 <MeetBot> Meeting started Tue Apr 25 17:58:05 2023 UTC.  The chair is Emperor. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:58:05 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
17:58:17 <Emperor> Hi folks :)
17:58:40 <Emperor> #topic Roll Call
17:58:43 <Emperor> Matthew Vernon
17:58:43 <Myon> Christoph Berg
17:58:44 <helmut> Helmut Grohne
17:58:51 <smcv> Simon McVittie
17:59:40 <Emperor> I'm guessing spwhitton is unwell enough to not be joining us, but let's give mjg59 a few minutes
18:00:53 <mjg59> Matthew Garrett
18:01:00 <Emperor> hi :)
18:01:13 <Emperor> #topic Review of previous meeting AIs
18:01:32 <Emperor> I did post to d-d-a about nominations, but I don't think we had any takers, alas
18:01:46 <helmut> I did the thing I was tasked to do.
18:02:05 <Emperor> thanks.
18:02:29 <Emperor> I think helmut's point on the agenda ordering was fair enough, so let's go with that.
18:02:37 <Myon> aye
18:02:40 <Emperor> #topic the moratorium on moving files into /usr
18:03:11 <Emperor> We issued advice to stop moving individual packages' files into /usr
18:03:30 <helmut> would someone remind me which problems exactly the moratorium targets? I know about the problem of moving files from / to /usr + moving files between packages. Any other problem?
18:03:43 <Emperor> and said we expected that advice would cease to be necessary at some point during the Debian 13 release cycle
18:03:45 <smcv> I believe that is the only significant problem it targets
18:04:06 <Emperor> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=994388#110 is the text in question
18:04:24 <helmut> I believe that I found a way to solve that problem in a different way just two hours ago in joint work with pochu
18:05:01 <smcv> that's great news. dpkg changes, or package changes, or what?
18:05:39 <helmut> I intended to post a long explanation to d-devel before our meeting, but really the prospective solution only came about just two hours ago in joint work with pochu
18:06:07 <Emperor> Fair enough - is there a TL;DR version for now so we have a rough idea?
18:06:11 <helmut> we took bluca seriously (in moving all the files post bookworm) and considered how to fix the moratorium issue
18:06:30 <helmut> and while breaks+replaces doesn't cut it, conflicts does
18:06:47 <helmut> so in 90% of the cases, moves just work and in the remaining 10% we can resort to conflicts
18:06:58 <smcv> "in 90% of the cases, moves just work" how so?
18:07:16 <helmut> except that this doesn't always work for a) complex upgrades where apt is unable to find a solution and b) essential packages
18:07:26 <helmut> smcv: those where we only move from / to /usr and not between packages
18:07:43 <Myon> so likely more 95 or 98%
18:08:13 <smcv> the thing that makes this transition particularly annoying is that the natural time to do it is, as bluca wants, at the beginning of a release cycle
18:08:33 <helmut> and in th 1% of caes where we have both move categories and cannot use conflicts, we can add a --no-rename diversion in preinst of the new package and remove that diversion in postinst to avert the removal
18:08:53 <smcv> but in mid 2023, when moving let's say /lib/systemd/system/dbus.service to /usr, we don't know yet whether it will become necessary to split/merge dbus before mid 2025
18:09:38 <helmut> smcv: yeah, right, but we can deal with that at that time as long as we always have a way forward. and the first step is breaks+replaces -> conflicts, and if that doesn't work, diversions are messy, but they'll work.
18:09:40 <Myon> helmut: heh, that's a cute idea indeed
18:09:46 <pochu[m]> that's alright, if that happens later you can then add the conflicts+replaces (or diversion if oldbin is essential)
18:10:23 <smcv> I think my point from -devel (and -dpkg) still stands, "not enough Breaks/Replaces" is already a RC bug, yet I frequently find myself reporting it
18:11:16 <smcv> "not enough Conflicts/Replaces or diversions" will equally be RC, but ideally we'd be detecting the need for it *before* it causes files to disappear
18:11:31 <Myon> that's often counter-intuitive, I wonder if dpkg couldn't do better there and be more forgiving
18:11:46 <helmut> Myon: please send thanks to pochu. I pondered this for like three days and then when I talked to pochu, it cleared up in like an hour
18:11:52 <mjg59> Do we have any tooling that automatically identifies cases where binaries have moved (a) between / and /usr and (b) between packages?
18:12:30 <helmut> mjg59: according to bluca, piuparts does that, but my understanding is that his piuparts patch does something different
18:12:30 <Myon> pochu[m]: thanks!
18:12:33 <smcv> if the file we're moving is something like /lib/systemd/system/dbus.service (or worse, /lib/*/libc.so.6) then finding out that we have the "it disappears" bug after it has made a testing/unstable user's system unbootable is kinda too late
18:12:41 <Emperor> smcv: +1
18:13:12 <smcv> having said that, there is value in "let unstable be unstable"
18:13:13 <Myon> smcv: we'd notice it before it goes to testing, I guess
18:13:24 <Myon> and unstable is, well, unstable
18:13:50 <helmut> regarding detection: I think that detecting the / -> /usr + package move scenario is relatively easily possible, not with piuparts, but with a new tool (dedup.d.n could be extended at relative ease)
18:13:59 <mjg59> If there's a technical mechanism that can be implemented in the packages which allows us to guarantee that these transitions occur safely, that seems fine
18:14:02 <Myon> plus that's only the dbus and libc maintainers that have to be careful, not the whole project
18:14:30 <smcv> well, dbus was just the first example that popped into my head of a systemd system unit
18:14:31 <helmut> mjg59: the question kinda is: is that diversion mechanism safe?
18:14:39 <smcv> (because it's the highest-popcon systemd unit I maintain)
18:14:40 <mjg59> But I'm not familiar enough with dpkg to have an immediate understanding of how this approach interacts with the failure modes
18:14:43 <Myon> smcv: s/dbus and libc/base system/
18:15:05 <mjg59> So I think I'd want to see a more detailed description of the interactions to be able to form an opinion
18:15:07 <Myon> we can hopefully expect these people to be careful
18:15:12 <mjg59> (Or I'm going to need to spend some time with the source)
18:15:14 <smcv> Myon: oh I see your point, only the packages in the boot -> desktop critical path need to worry about whether they broke that critical path
18:15:20 <helmut> I shall have a long version posted to d-devel today
18:15:25 <mjg59> helmut: Awesome, thank you
18:15:34 <smcv> and e.g. with my openarena-server maintainer hat on, I don't have to be that careful
18:15:49 <pochu[m]> for unstable we only have lintian autorejects, and we can't do it in lintian. But it could be easily done in an external service that files RC bugs as necessary, so testing could be kept safe
18:15:59 <helmut> the other point regarding detection is to figure out when we cannot do conflicts. I don't have a good idea on that one
18:16:37 <smcv> helmut: am I right in saying that Conflicts are always safe in terms of not having this bug, but they can cause the upgrade path to become impossible?
18:16:37 <Emperor> So from a making-a-decision point of view, are we at the point where we want to hold off deciding on the moratorium until helmut & pochu's work has been explained and reviewed a bit? Our next meeting is likely before the bookworm release?
18:16:47 <pochu[m]> the problem I see with using conflicts is with apt finding an upgrade path to trixie
18:17:04 <helmut> smcv: yes plus the issue that you cannot use conflicts to move files between essential packages
18:17:12 <pochu[m]> but I'm not too familiar with resolvers to anticipate if that's going to be too problematic
18:17:27 <smcv> I think the bar is (should be) higher for the amount of care taken by Essential maintainers
18:17:44 <Myon> perhaps the moratorium could be narrowed to say "don't move between /usr and packages in the base system" with the idea that anything outside will likely be resolvable by apt?
18:18:12 <Myon> and that would be removed after trixie
18:18:27 <smcv> Myon: that would disallow doing this automatically as packages get rebuilt, e.g. in debhelper, if I understand correctly
18:18:38 <Emperor> Myon: that seems like the sort of thing that might fall out of a more detailed discussion of the proposal?
18:18:54 <Myon> smcv: that's only the / -> /usr move, not between packages, so not a problem
18:18:59 <helmut> at this time, I'm in favour of deferring the question of the moratorium due to all the news
18:19:16 <smcv> helmut: +1
18:19:19 <Myon> Emperor: yes, just thinking ahead
18:19:23 <Emperor> sure
18:19:38 <helmut> if the projected diversion hack actually works, I'm in favour of explaining the solution and dropping the moratorium immediately
18:19:45 <Myon> I think we are beginning to see a good plan
18:19:52 <Emperor> any opposition to deferring a decision on the moratorium until the May meeting?
18:20:13 <Myon> Emperor: I'd need time to think about that anyway
18:20:15 <smcv> what I had expected to come into this meeting and say was: our moratorium is strictly speaking redundant with the release team's, but for clarity we should extend ours into trixie too
18:20:44 <mjg59> Scratch proposal: If rough consensus on d-d after this proposal is described is that it satisfies our previously described concerns, the moratorium can be dropped without explicit t-c action?
18:20:46 <helmut> if you encounter any issues the projected idea, please reach out to me immediately. it may still have a fatal flaw and become a no-brainer
18:20:48 <Emperor> smcv: I think that was roughly my position too
18:20:55 <smcv> but if helmut and pochu[m]'s plan is as promising as it seems to be, then ... yes, what Myon said
18:21:04 <smcv> *what mjg59 said
18:21:12 <Emperor> mjg59: Mmm
18:21:15 <smcv> thanks, tab completion. invaulable
18:21:24 <Myon> Mjon :)
18:21:56 <helmut> I mean this is like 3 hours old now. it seems too good to be true.
18:22:00 <Emperor> #agreed revisit moratorium on moving files into /usr at May meeting in light of discussion around helmut and pochu's proposal
18:22:11 <Emperor> #topic DEP17
18:22:25 <Emperor> My feeling on this is that the TC doesn't need to get involved at this point
18:22:42 <helmut> the main reason for doing DEP17 is to address the issues of the moratorium
18:22:54 <Emperor> (notwithstanding the impact of the previously-discussed work)
18:23:20 <helmut> so any discussion on DEP17 must be based on the assumption that the previous idea with conficts + diversions doesn't work for some reason
18:23:33 <mjg59> If we can solve this problem through package policy rather than doing significant engineering work in dpkg, that seems preferable
18:24:08 <Emperor> +1
18:24:11 <pochu[m]> mjg59: agreed
18:24:13 <helmut> it also must be based on the assumption that for the reason of conflicts and diversions not working, we continue imposing the moratorium until dpkg is fixed
18:24:35 <mjg59> But I'm kind of reluctant to just say "Let's not talk about this" because if the new proposal doesn't work out, we should have some roadmap for this as soon as possible
18:24:40 <helmut> so we're in a rather hypothetical land at this point
18:25:13 <smcv> there are a couple of things that I keep thinking about DEP17 and related dpkg work
18:25:15 <mjg59> Although if we aim to hold May's meeting on the original date (ie, two weeks from now) maybe that's a reasonable compromise?
18:25:18 <Emperor> mjg59: next meeting is in 2 weeks, which isn't that long to punt this for; and by then we may well have a clearer idea of where we stand
18:25:21 <Myon> mjg59: we still need dpkg engineering to fix `dpkg -S` even if all packages get updated to /usr
18:25:45 <helmut> Myon: why?
18:26:09 <Myon> because dpkg -S /bin/bla doesn't find /usr/bin/bla
18:26:15 <Myon> (workaround: dpkg -S bin/bla)
18:26:45 <smcv> helmut: if your scheme from today works, am I right to think that your intention is to keep dpkg -S the same as it currently is, only finding files by their "canonical" path as seen in the data.tar?
18:27:19 <Emperor> ...and will dpkg -S $(which foo) produce the right answer?
18:27:20 <smcv> which hopefully by the time we release trixie would always start with /etc, /usr or /var for software in Debian?
18:27:33 <helmut> I consider the -S issue non-rc, so yeah, I'd leave it as is and leave it to others to fix when they see fit (or whether)
18:28:01 <Myon> non-rc, yes, but users will still hate us for it
18:28:12 <mjg59> smcv: I think assumptions like "/bin/sh exists and is resolvable via dpkg" aren't obviously unreasonable
18:28:41 <pochu[m]> but if we move everything into /usr after the moratorium is lifted, then that's not much of an issue is it?
18:28:42 <helmut> yeah, in trixie the goal should be that all files ahve been moved to their canonical location and since $PATH usually lists /usr/bin before /bin, dpkg -S $(which foo) should work
18:28:48 <Emperor> on my bookworm system dpkg -S $(which bash) fails, which is pretty poor
18:28:52 <Myon> are we saying the solution is to remove /bin from the default PATH?
18:29:16 <smcv> Myon: e.g. Arch did
18:29:36 <Emperor> Myon: ATM I think /usr/bin comes before it anyway
18:29:46 <helmut> I expect /bin to be removed from $PATH eventually
18:29:52 <Myon> nod
18:30:00 <smcv> my $dayjob work on making Steam stuff cross-distro-portable makes me painfully familiar with distro quirks like that
18:30:02 <Myon> still, `dpkg -S /bin/sh` should continue to work
18:30:22 <Myon> but I made us digress
18:30:31 <smcv> if we support chrooting into an older or non-Debian environment, then rushing to remove /bin from PATH would be unfriendly
18:30:35 <Emperor> Mmm, I was wondering if I should try and drag us back to topic :)
18:30:39 <helmut> so back to DEP17, do you consider this to be a reasonable plan-b?
18:31:00 <smcv> I have a couple of observations I want to make
18:31:43 <smcv> one issue that keeps coming up in DEP17 and similar efforts is that dpkg's structure (and dpkg's maintainer's dislike of usrmerge) encourages interfaces like this one to be general
18:31:54 <smcv> which then makes them API surface area
18:32:22 <smcv> I think it would be good to be clear about whether an interface is intended to be sysadmin-facing API, or whether it's only intended to be distro-integrator-facing
18:32:40 <helmut> do you see any way to avoid them being API surface? I was assuming that this would be inherent in the problem at hand.
18:32:59 <smcv> for example if I offload /usr/share/games onto /srv/more-space-on-this-disk/games, am I expected/encouraged/discouraged to register that alias with dpkg?
18:33:27 <smcv> (IMO: no, only distro integrators should be using this interface)
18:33:40 <helmut> vaguely agreed
18:33:44 <smcv> (and also I should be using a bind-mount of course)
18:33:57 <Myon> why does it have to be API when we are just talking about a handful of fixed locations?
18:34:04 <smcv> because of this loop:
18:34:24 <smcv> <contributor> ok so I want to add code to handle merged-/usr
18:34:46 <smcv> <maintainer> I don't like merged-/usr, add a general-purpose feature or nothing
18:34:47 <ansgar> Myon: Becuase people make it more generic to cover /etc (hi, conffiles!) and include revert-from-usrmerge-to-non-usrmerge-in-case-omnious-sponsors-disapper and support-non-usrmerge-Debian-based-distros.
18:34:54 <helmut> Myon: 1) because dpkg is used by many projects some of which are debian derivatives 2) because dpkg can be used to operate on an (unmerged) installation via --root and thus the merged-ness must be a property of the installation not a property of dpkg
18:35:14 <Myon> helmut: that's still a boolean flag in a config file
18:35:18 <smcv> <contributor> ok, here's a general-purpose interface to handle arbitrary aliasing
18:35:33 <smcv> <maintainer> now this is "API" that I have to support forever, don't do that
18:35:40 <ansgar> Myon: It's not a configuration option, so it should not be a configuration file.
18:35:51 <Myon> smcv: yes, I'm saying we should break the loop at 1)
18:35:56 <smcv> either literally that, or a perception that there would be that
18:36:31 <helmut> smcv: I think I understand the problem you describe. could you more explicitly note your conclusion from that?
18:37:00 <smcv> my conclusion is that people perceive (rightly or wrongly) that there is no contribution they can make that would be acceptable
18:37:32 <helmut> that seemed more of the problem part to me. I suppose you don't have a solution in mind
18:37:36 <smcv> because a generic version imposes a support cost on the maintainer forever, and a specific version is giving dpkg code to do something its maintainer thinks it shouldn't do
18:37:52 <smcv> correct, I don't know what the answer to this is
18:38:11 <smcv> if I did, I'd have suggested it already
18:38:24 <helmut> still your descriptions increases clarity of the problem. thanks
18:38:26 <Emperor> it is a bit "well, I wouldn't start from here"
18:38:33 <smcv> but this relates to the other thing I wanted to put into the record of the meeting
18:38:55 <Myon> helmut: if --add-alias only records the value (and doesn't modify anything else), you don't need that --option, and can just put a list in a config file
18:39:00 <smcv> which is: dpkg is a native package, which in theory means it does what Debian wants, no more no less
18:39:03 <Myon> perhaps that would make it less API-ish?
18:39:22 <smcv> but at the same time, dpkg maintenance in some ways behaves like an upstream
18:39:42 <smcv> analogous to let's say rpm.org being upstream from all the RPM-based distros
18:39:44 <helmut> smcv: guillem sees debian uploads as upstream releases, and yeah this is a problem
18:39:49 <smcv> and we kinda can't have it both ways
18:40:20 <smcv> if it's an upstream, then Debian needs to be able to apply delta if the cost exceeds the benefit, the same as we do for dbus
18:40:23 <helmut> Myon: then that config file becomes the api, no? and then you'd have to think where to store it (not /etc)
18:40:38 <smcv> (for a while I was applying delta to Debian's dbus that I had explicitly rejected with my upstream hat on!)
18:41:15 <helmut> smcv: in principle, I'd agree with you, but my impression is that having this discussion for real would shrink the dpkg team
18:41:20 <Myon> helmut: yes, but it's perhaps more an internal detail
18:41:36 <ansgar> helmut: We once had a larger maintainer team for dpkg...
18:42:36 <Emperor> OK, is there more we think we should discuss re DEP17 today?
18:42:40 <helmut> Myon: to me, that is a different color of the same shed. I'm not opposed to changing the color, but the implementation is the same
18:42:53 <mjg59> I think I'm fine punting DEP17 to next meeting assuming we expect to hold it 2 weeks from now
18:43:00 * Emperor still mindful that we might be able to have a less hypothetical discussion in a couple of weeks
18:43:14 <smcv> yeah that's the outcome I'm hoping for
18:43:17 <helmut> Emperor: well, I don't perceive anything as an answer to the question of whether dep17 is something we could move forward as a plan-b.
18:44:01 <mjg59> I'm not sure we'll be able to have a more useful conversation about DEP17 in two weeks
18:44:09 <mjg59> But we may be able to just not have that conversation
18:44:10 <helmut> mjg59++
18:44:13 <Emperor> that
18:44:31 <mjg59> So not spending more time on it now seems fine
18:44:38 <Myon> +1
18:45:05 <Myon> (though it would be a pity if a workaround (yay!) would mean dpkg remains unfixed)
18:45:10 <smcv> to the extent of my understanding of dpkg (which is not very much in terms of finer technical detail), DEP17 seems like a viable plan b, modulo some wrangling about exactly how the "/bin aliases /usr/bin" is stored/represented
18:45:28 <mjg59> The broader "Is dpkg maintained in the best way possible for Debian's benefit" question is one that may be outside our remit
18:45:32 <helmut> Myon: once files are moved from / to /usr, nothing is aliased anymore, so dpkg doesn't need any fix anymore
18:45:39 <Emperor> mjg59: yes
18:45:54 <Myon> helmut: /bin/sh will be a thing forever
18:45:58 <Emperor> (and I think if we do have to get really into the details of dpkg's operations, I'd find it easier to do a bunch of reading first)
18:46:08 <helmut> Myon: so be it
18:47:18 <Emperor> #agreed revisit DEP17 at the May meeting iff necessary
18:47:21 <Emperor> #topic AOB
18:47:42 <Myon> not here
18:47:58 <Emperor> I should note my apologies for the 9th May meeting (I'm away for a couple of weeks, so there are not obvious alternative dates, so I don't suggest moving the date)
18:48:07 <Emperor> ...but I have no further things for today
18:48:33 <smcv> I hear we have a provisional release date! So if you have (or want to have) last minute bookworm fixes or post-trixie plans, now is the time
18:48:46 <smcv> *post-start-of-trixie
18:49:25 <Emperor> oooh
18:50:10 * Emperor is going to make themself popular by bumping the "you're still using the obsolete pcre3 library" bugs to RC once bookworm is out
18:50:31 <Emperor> But, that's way off-topic. Thanks for your time folks, and I hope spwhitton is feeling better soon :)
18:50:34 <Emperor> #endmeeting