16:01:00 <vagrantc> #startmeeting
16:01:00 <MeetBot> Meeting started Tue Jun 19 16:01:00 2018 UTC.  The chair is vagrantc. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:01:00 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
16:01:12 <vagrantc> #topic hello
16:01:30 <vagrantc> if folks could say a little hello so we know who's here
16:01:38 * yashsriv is here
16:01:42 * sangy waves
16:02:33 <vagrantc> let's wait a minute or two, see if others can show up
16:02:41 <jelle> o/
16:05:40 <vagrantc> alright, i'm guessing that's it!
16:05:51 * sangy shrugs
16:05:59 <vagrantc> lamby marked themselves in apologies, so i'm guessing won't make it
16:06:30 <vagrantc> a meeting reminder via email 2-3 days in advance is kind of key, sorry i didn't think of it
16:06:46 <sangy> I kinda rushed it, because I wanted to get back to them sooner rather than later.
16:06:55 <vagrantc> #topic progress on debian rebuilder
16:07:07 <sangy> yashsriv: this is you :)
16:07:08 <yashsriv> hi
16:07:10 <vagrantc> yashsriv: you have the mic!
16:07:48 <yashsriv> so the script I was working on is nearing completion. It manages to take a buildinfo file as input and then perform the complete build
16:08:11 <vagrantc> any links to it?
16:08:24 <yashsriv> https://salsa.debian.org/yashsriv-guest/debian-rebuilder-setup/blob/integrate-srebuild/builder/srebuild
16:08:51 <vagrantc> #link https://salsa.debian.org/yashsriv-guest/debian-rebuilder-setup/blob/integrate-srebuild/builder/srebuild
16:09:13 <mapreri> o/
16:09:18 <sangy> I think the idea with this is that any interested orgs can use this ansible playbooks to deploy a re-builder and attest for its reproducibility.
16:09:19 <mapreri> (sorry for the late arrival)
16:09:20 <sangy> mapreri: o/
16:09:40 <sangy> on our side, NYU will deploy a rebuilder and sign any artifacts it reproduces :)
16:09:47 <vagrantc> very cool
16:10:21 <yashsriv> So the builder was the very first step. The next step is setting up a scheduler to schedule these builds from time to time and an interface for exposing the end results
16:11:02 <jelle> no meetbot?
16:11:04 <mapreri> yashsriv: do you think it would be trivial enough to have it run pbuilder instead?
16:11:07 <sangy> jelle: ye there is
16:11:39 <jelle> oh lol, I'm blind sorry.
16:11:56 <yashsriv> If pbuilder supports prebuild hooks, then should be trivial enough. I had looked into both but then theses scripts already existed so I got biased.
16:12:34 <vagrantc> it might be good to have the infrastructure reproducing builds in the archive using sbuild, since this is more similar to the official buildd machines
16:12:49 <mapreri> yashsriv: pbuilder supports many more hooks than sbuild, afaik :)  https://manpages.debian.org/stretch/pbuilder/pbuilder.8.en.html grep for --hookdir
16:12:58 <vagrantc> as well as getting the other main tool used to build packages working with sbuild
16:12:58 <mapreri> but yes, also that
16:13:31 <mapreri> can somebody remind me where's the agenda?
16:13:39 <sangy> https://pad.riseup.net/p/reproducible-irc-meeting-12
16:13:41 <sangy> mapreri: ^
16:13:45 <mapreri> ta
16:13:54 <yashsriv> regarding  the scheduler, I had a doubt. The buildd servers can be used to schedule for every new package entering if I'm not wrong.
16:14:17 <yashsriv> *information from buildd
16:14:17 <mapreri> define "build servers" there
16:14:52 <vagrantc> yashsriv: you might need to wait till things land in the archive
16:15:26 <vagrantc> yashsriv: as well as we still don't have the actual .buildinfo content exposed anywhere
16:15:33 <vagrantc> anywhere public
16:15:48 <yashsriv> http://buildinfo.debian.net ??
16:15:57 <vagrantc> the official builds aren't published there
16:16:19 <mapreri> (yet, there is an open bug somewhere)
16:16:30 <vagrantc> sure, we'd like to fix that, of course
16:17:01 <sangy> and yeah, keyword "yet". I'm assuming using buildinfo.debian.net for testing/development is correct?
16:17:01 <vagrantc> yashsriv: i wouldn't block your work on that part just yet
16:17:21 <mapreri> sangy: I'd say so, yes
16:17:30 <vagrantc> sure, for testing the infrastructure, buildinfo.debian.net should work for now
16:17:50 <mapreri> hell, the very goal of buildinfo.d.n was to expirement in a thing sharing proof of builds (i.e. the .buildinfo) :)
16:18:19 <yashsriv> does buildinfo.d.n have a machine readable interface as well?
16:18:22 <vagrantc> #link https://bugs.debian.org/763822
16:18:25 <sangy> so what I was thinking about is that these buildinfo files get pushed out, and they get pulled by (e.g., NYU) to schedule a rebuild locally. Finally they'd push their own buildinfo (and signed metadata)
16:18:33 <vagrantc> yashsriv: yes
16:18:38 <mapreri> yashsriv: it has a fair api, yes
16:19:00 <vagrantc> #link https://bugs.debian.org/862073
16:19:03 <jelle> I was thinking for arch of rebuilding based on a package (pkg.tar.xz) the .deb equivalent
16:19:13 <mapreri> well
16:19:31 <vagrantc> yashsriv: and if the API is missing anything you need, lamby is generally responsive to bug reports
16:19:35 <sangy> jelle: I think that can happen with arch, since the .BUILDINFO is packaged in the .tar.xz
16:20:03 <jelle> sangy: but we can also start publishiing the BUILDINFO, since it contains pkgname, pkgbase and pkgver and pkgarch
16:20:15 <mapreri> yashsriv: the /api/ endpoint only has the submission thing, but there are other urls to get .buildinfo quite programatically.  agian, expanding it is not a issue.
16:20:30 <sangy> jelle: well that'd be nice. We don't have a working archive yet though. (do we want to add that as a meeting topic? :P)
16:20:46 <yashsriv> perfect. So now what should we aim to expose once we got something built? As someone mentioned yesterday, BUILDINFO files aren't necessarily exactly the same
16:20:58 <jelle> sangy: the upside of using a .tar.xz is that we can verify it though. So does debian plan to sign these BUILDINFO files?
16:21:04 <yashsriv> so we'd probably need to expose the SHA sums of the end products?
16:21:30 <sangy> yashsriv: I'm biased, but in-toto link metadata would tie things very nicely in this case
16:21:31 <vagrantc> jelle: buildinfo files should always be signed
16:21:40 <jelle> ok :)
16:22:31 <mapreri> yashsriv: I'd just POST your signed buildinfo to buildinfo.d.n and have it deal with the next steps
16:22:47 <mapreri> buildinfo.d.n already has a concept of "reproducible" and "unreproducible", although sometimes not perfect
16:24:04 <sangy> well, I guess that settles it? action point -> try to integrate with pbuilder for scheduling etc?
16:24:10 <vagrantc> the main problem being is it has know way to know which .buildinfo's should reasonably know which other .buildinfo files should be compared for reproducibility
16:24:43 <vagrantc> sangy: wait ... where did that come from? :)
16:24:46 <mapreri> sangy: integrate with pbuilder was more so like I can use it without learning another tool :3  but from what I saw of that script it shouldn't be hard to do at all
16:25:01 <sangy> oh ok ok :P
16:25:05 <mapreri> adding a scheduler, well… that's another point.
16:25:06 <sangy> vagrantc: from there
16:25:33 <vagrantc> the intention of this is something entirely independent from jenkins.debian.net ?
16:25:38 <sangy> vagrantc: re: how to know which .buildinfo files to correlate. I *do* ferviently believe that in-toto link metadata would make it easier
16:25:42 <vagrantc> er, tests.reproducible-builds.org
16:25:55 <sangy> vagrantc: that's what h0lger intended, yes. As it'd be more lightweight and one-click-deployment for interersted organizations
16:26:10 <vagrantc> yeah, i think this came out of discussions at the last summit
16:26:18 <mapreri> tbh, I've been hacking around the t.r-b.o infra in the recent past, I feel like I'm near enough to make it less jenkins-y and more non-debian-friendly, which could easily include integrate it with the rebuilder.  Alas, not so easy this last part.
16:27:19 <vagrantc> while i see the advantages of everything all being in one place, i think it's best to not have to integrate everything in one place to experiment with a few different methods
16:27:23 <mapreri> but if it lives by itself, it can always be plugged in anywhere, so :)
16:27:38 <mapreri> ack
16:27:45 <vagrantc> especially at this point
16:27:58 <vagrantc> so i like the idea of something lightweight and "simple" :)
16:28:42 <vagrantc> so, do we have any actionable items for this topic? any further things that need discussion?
16:29:11 <sangy> jelle: did you want to bring up the arch-archive stuff or hosting the buildinfo's somewhere?
16:29:17 <vagrantc> jelle: and did you want to add some talk of an archlinux rebuilder as a separate agenda item, or did you get enough from the discussion so far?
16:30:32 <jelle> vagrantc: we can discuss the arch one, so far the 'my plan' is to make users able to verify packages
16:30:56 * mapreri has to go afk, sorry for leaving unexpectedly!
16:30:58 <vagrantc> shall we move on to that as a topic?
16:31:05 <jelle> sure
16:31:17 <vagrantc> #topic verifying archlinux packages
16:31:34 <sangy> jelle: there was a plan to have .BUILDINFO files be hosted on archweb no?
16:31:48 <vagrantc> also, if anyone has any additional items for the agenda, just let me know, we've got very little scheduled on the agenda
16:32:23 <jelle> might be good to give a little status update, we now can create reproducible packages with the latest released pacman. This also includes an updated BUILDINFO file. We have worked on creating a script to not remove packages which are mentioned in a BUILDINFO file in our repo, since our archive doesn't have infinite space.
16:32:34 <jelle> This has led to a rebuild of old packages without BUILDINFO files!
16:32:58 <jelle> sangy: we could do that, but archweb would provide BUILDINFO files unsigned, which then might defeat the point?
16:33:19 <sangy> jelle: I don't see why can't the developer sign the buildinfo files?
16:34:26 <jelle> sangy: well, archweb would read updated .pkg.tar.xz's and extract BUILDINFO files
16:34:55 <sangy> right, which could be signed by develoeprs
16:35:02 <vagrantc> the ideal for .buildinfo files is that multiple independent parties sign separate .buildinfo files which can then be used to corroborate each other
16:36:01 <jelle> hmmmm
16:36:30 <vagrantc> (or raise questions about the results, if they don't match)
16:36:38 <jelle> hehe
16:37:14 <sangy> jelle: so is the reason they aren't being signed right now is becuase they are embedded in the .tar.xz which is signed?
16:37:39 <jelle> sangy: yes, well and they aren't separate files only included in our packages
16:38:02 <jelle> on archlinux.org we could extract them and put them on our website, but then they aren't signed.
16:38:06 <vagrantc> jelle: the .pkg.tar.xz includes what, typically? (sorry for my arch ignorance)
16:38:27 <jelle> vagrantc: the actualy package data, an some metadata
16:38:46 <sangy> vagrantc: the pkg itself (data), a .SRCINFO file and an .MTREE file, now there's a .BUILDINFO file too
16:39:08 <sangy> (not so sure about the .SRCINFO actually)
16:39:27 <vagrantc> hm. embedding the .buildinfo means you actually have to extract the archive to verify the checksums on the rest of it, then?
16:39:50 <sangy> yes
16:40:16 <sangy> the argument for that is that you want to check against it anyway
16:40:54 <vagrantc> makes bit-for-bit reproducibility of the .pkg.tar.xz basically impossible...
16:41:36 <vagrantc> on the plus side, you can actually distribute the .buildinfo that you're actually using for that particular build trivially
16:42:52 <vagrantc> or rather, the .buildinfo used to produce the rest of the .pkg.tar.xz
16:43:58 <jelle> yup
16:44:03 <vagrantc> so, you could distribute third-party .buildinfo files through some other out-of-band mechanism, which are signed
16:44:31 <sangy> that'd be ideal. And having the original buildinfo signed by the packager would also be good
16:44:33 <jelle> vagrantc: yup that is still possible
16:44:46 <raboof[m]> (late 'hello' ;) )
16:44:53 <jelle> btw, we could change our upload process, that just requires more work :)
16:45:08 <jelle> personally I think it might be a bit strange to put it unsigned on our website.
16:45:22 <vagrantc> yeah, i wouldn't bother extracting an unsigned .buildinfo
16:45:32 <sangy> jelle: I still don't see why can't we sign it before putting it in the .tar.xz for the time being
16:45:56 <vagrantc> do the uploaders upload source-only? or do they upload binaries?
16:45:57 <jelle> sangy: how would you reproduce it then ;-)
16:46:02 <jelle> vagrantc: we upload binaries
16:46:09 <vagrantc> oh.
16:46:15 <jelle> we sign the .pkg.tar.xz locally, sign it and upload it
16:46:20 <vagrantc> then they *definitely* should sign them! :)
16:46:56 <jelle> well yeah it can be done, just requires changes and extracting them from the .pkg.tar.xz :)
16:47:47 <sangy> jelle: I'm down for working on that, but I have a lot in my plate right now :S. Any updates on the archive?
16:48:04 <jelle> my focus right now, is getting a script together so users can reproduce based on a .pkg.tar.xz and then later we can start thinking about publishing :)
16:48:33 <sangy> yeah, I think without archive we can't really reproduce anything :P
16:48:36 <jelle> sangy: ah, yes, our archive needs fixing, since now we archive once a day and might miss packages depending on a package which was updated in that timeslot. So twice on the same day.
16:48:46 <jelle> sangy: well we have the archive, just one small issue.
16:49:47 <sangy> ack, nice :)
16:50:21 <jelle> anthraxx: created a script to remove old packages which are not referenced in any of our repo packages
16:51:29 <jelle> if we want to re-produce every package we ever had in our repository we need more space :)
16:51:36 <sangy> yeah I remember the discussion about those spureous mylinuxkeyring packages
16:51:55 <sangy> and yeah :[
16:52:38 <jelle> yup, we also had packages with a BUILDINFO referencing packages which are not in our repo!!!!! This is still an issue which we need to resolve, basically disallowing such an upload
16:53:08 <sangy> right. I mean that's the nice thing about this, it catches stuff like this :)
16:54:11 <raboof[m]> I'm not too up to speed with the state of things, would that archive also be the place for third parties to upload their signed buildinfo files (for cross-checking)?
16:54:15 <sangy> well, so I don't know. I guess action point -> sign the .BUILDINFO? :D
16:54:31 <sangy> raboof[m]: no, I think that'd be another platform (similar to buildinfo.d.n)
16:54:54 <jelle> raboof[m]: no, it's an archive of our old packages required to reproduce the repository packages (since rolling release)
16:55:00 <raboof[m]> that sounds like a sensible thing to do in any case, looking at the above, yes
16:56:59 <vagrantc> due to time, maybe we've had enough on this topic for the meeting? can keep discussing later, of course :)
16:57:38 <jelle> sure!
16:57:56 <vagrantc> #topic next meeting
16:58:01 <raboof[m]> I might be up for collaborating on that repo for signed buildinfo files (either based on http://buildinfo.debian.net/ or a standalone thing) - though I have limited time as well :)
16:58:06 <vagrantc> so, so we have a date for the next meeting?
16:58:33 <vagrantc> and also, maybe someone to commit to sending reminder emails 2-3 days in advance?
16:58:45 <sangy> vagrantc: I can do that
16:59:07 <sangy> vagrantc: and should we schedule same tuesday a month from now? I'd like lamby and h01ger to be able to make it though...
17:00:11 <vagrantc> can't speak to their availability, but otherwise same time and day of week a month from now seems reasonable
17:01:04 <sangy> ok, should we set that date tentatively then?
17:01:22 <vagrantc> i think so, and leave it open for revolt :)
17:01:46 <sangy> then let's do Tuesday 17th of July on 16:00 UTC  :D
17:01:48 <vagrantc> looks like the 17th? 16UTC ?
17:02:06 <raboof[m]> perfect
17:02:44 <sangy> +1
17:02:52 <yashsriv> 👍
17:03:09 <vagrantc> #agreed tentative date for next meeting 17th of July, 16:00 UTC.
17:03:10 <sangy> I'll send the reminder on the 26th then. Possibly another one the week after
17:03:26 <vagrantc> #commit sangy to send reminders
17:03:54 <vagrantc> ok, thanks all!
17:03:56 <vagrantc> #endmeeting