18:02:26 #startmeeting Tor Browser Release Meeting 2022-03-7 18:02:26 Meeting started Mon Mar 7 18:02:26 2022 UTC. The chair is donuts. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:02:26 Useful Commands: #action #agreed #help #info #idea #link #topic. 18:02:34 I think I did that right 18:02:56 you should copy the command like Richard :D 18:02:58 gaba is afk today so we should start without her 18:03:00 you did it :3 18:03:08 haha yeah I have to look it up every time 18:04:01 looks like someone (aguestuser?) is adding things to the pad 18:05:23 indeed! 18:05:48 okay let's get started 18:06:30 11.0.7 stable is getting released tomorrow, but desktop only? 18:06:42 bolkm is already signing 18:06:46 perfect 18:06:50 so I don't think we have time to add also Android 18:06:54 yeah 11.0.7 can ship once boklm gives the thumbs up 18:06:59 and 11.5a6 is planned for next week 18:07:05 or more likely, he will just make it happen 18:07:07 yeah I was going to ask what the status is with android releases 18:07:11 i'm working on blog post for 11.0.7 now 18:07:22 donuts: but we may release 11.5a6 earlier, because of the security patches 18:07:31 donuts: did you see that 11.5a5 went out with the download bugfix? 18:07:39 yeah 11.5a6 should go out asap after 11.0.7 18:07:51 we've already built it (at least me, I think that boklm is going to build it soon, if he's not already doing it) 18:08:09 aguestuser: ah no, I missed that! on Thursday? 18:08:21 that's great, I'll bring it up at tomorrow's UX team meeting to test 18:08:33 https://blog.torproject.org/new-release-tor-browser-115a5/ 18:08:38 oh one thing I wanted to call out, I've added a 'Release Prep' tag 18:08:48 what are the security patches for 11.5a6 btw? 18:09:00 ty aguestuser 18:09:07 to Tor Browser issues on gitlab, and you can see all of our planned/closed release tracking tickets 18:09:22 (and there's a new column at the end on the defualt Tor Browser board in gitlab) 18:09:44 donuts: the same that we added to 11.0.7 (exactly the same, because both are based on 91.7) 18:10:16 commits are tor-browser@fe4d6e1d236831635d76f062e013c6bf25d58408 and tor-browser@1794b76288c6fa7fed6f4caae88335ed57145e65 18:10:54 richard, PieroV: the linked mozilla bug is private so it's a bit hard to understand what attack they are preventing 18:10:59 (particularly if i'm donuts trying to explain this to users) 18:11:08 do we have anything public from moz explaining the bug? 18:11:19 richard: it looks like release prep is in applications/tor-browser only, was that intentional? 18:11:23 we have a CVE 18:11:25 or should it be on all of applications? 18:11:31 two CVEs, actually 18:11:55 PieroV: do we have links? (i looked at all our tickets associated with the bugfix and couldn't find any public links) 18:11:55 CVE-2022-26485 and CVE-2022-26486, let me find if I can find a statement from Mozilla 18:11:58 donuts: yeah i think that's a fine place for it? 18:11:59 yeah anything public you could point me to would be great, although it hasn't came up yet as far as I'm aware 18:12:05 and was too lazy to scrollback in #tor-internal ;) 18:12:32 https://www.mozilla.org/en-US/security/advisories/mfsa2022-09/ 18:12:33 https://www.mozilla.org/en-US/security/advisories/mfsa2022-09/ 18:12:38 heh. too slow. 18:12:44 ah fantastic, ty both :) 18:13:03 love these release prep tickets btw richard 18:13:29 yeah I'm going to add an issue template to help track all the work that goes into it as well 18:13:32 richard: speaking of which... i am supposed to make contributions to the release prep issue template... but i can't find the template 18:13:37 doh 18:13:38 for tracking+documentation purposes 18:13:42 messages stepped on each other 18:13:44 aguestuer: yeah it doesn't exist yet ;) 18:13:58 also nice touch adding it to the kanban 18:14:13 richard: but i'm guessing i can look at your last one and get the gist? (for my next release prep ticket?) 18:14:24 hopefully i'll have an MR ready for it this week but you know 18:14:34 nw 18:14:34 :) 18:14:38 aguestuser: so currently that ticket just links the relevant issues for a given release 18:14:59 but I'm *going to* add a checklist for rebase, version checks, signing tagging, etc 18:15:21 coool! :) 18:15:21 anyway i'll ping you when i have something 18:15:24 ack 18:15:45 okay so: what discussion point are we on now? 18:15:46 ;) 18:16:07 Android releases then the discussion items? 18:17:19 i had added android releases as a discussion item 18:17:31 but am happy to jump to that if folks think it's prio 18:17:54 i mainly want to figure out what numbers to give things 18:18:01 and when to target them for release 18:18:08 so i can update calendar and issues accordingly 18:18:23 decided today: will begin rebasing onto v99 *now* 18:18:31 dope 18:19:14 we need another stable release this month to get the fenix v96 update into stable 18:19:20 but we just shipped it to alpha last week 18:19:34 so if we're trying to get on an "every 2 week" cadence 18:20:02 every two weeks for a new android stable? 18:20:03 i was thinking of doing the first android release this month as an alpha release rebasing onto v99 18:20:05 then a stable 18:20:09 atm it looks like android and desktop releases are kind of leapfrogging each other on the calendar, is there a plan to get desktop and android releases in sync again? 18:20:28 so we've tentatively agreed to interleave tor-browser desktop and android releases (ignore chemspill/emergency releases) 18:20:32 I think it was intended 18:20:38 donuts: i had made the calendar assuming we wanted to "leapfrog" 18:20:42 oh gotcha 18:20:45 yeah 18:20:47 (it is admittedly somewhat confusing to try to do that!) 18:20:55 otherwise it's too much to build/sign all at one time 18:21:05 the version numbering seems a little weird then 18:21:16 but idk how that works 18:22:03 yeah i'm not sure how to best resolve that 18:22:14 if now's an okay time, i'd love to touch base on logistics of interleaved releases 18:22:24 shoot 18:22:26 i tried to fill in release dates in the calendar for ~2 months out 18:22:48 with the goal of doing android builds in the weeks in-between when desktop builds were happening 18:22:49 BUT 18:23:08 aguestuser: (just fyi, we want to get onto a bi-weekly release schedule, but that shouldn't delay putting out a stable version while we're still behind Mozilla) 18:23:25 sysrqb: wait i really wanna hear that 18:23:32 but also want to communicate another thing 18:23:42 sysrqb: wait now i'm confused 18:23:52 i mapped out a bi-weekly release schedule in the calendar 18:23:55 is that not what you are talking about? 18:24:02 err *we*? 18:24:19 yes, but we're month's behind mozilla 18:24:25 i was trying to alternate an alpha release with a stabel release every 2 weeks 18:24:33 and put that into the calendar from today until 2 months from now 18:24:37 but had a hard time doing it neatly 18:24:50 so, in this case, where we have what-appears-to-be a working version based on Moz96 18:24:52 b/c dekstop has a ~1 week lag between signing/tagging alpha releases 18:24:56 and shipping them 18:25:00 we shouldn't delay putting out a stable just because we want a biweekly cadence 18:25:02 gah 18:25:06 this is really hard to follow 18:25:11 can we have a "talking stick" or something? 18:25:22 ok 18:25:47 sysrqb: can you outline how this bi-weekly release schedule should look for android? 18:25:49 aguestuser: you can go ahead, i'll stop 18:25:58 sysrqb: okay, so i *think* thing (1) is that we want to prioritize stable 18:26:10 like that's the very next thing b/c playing catch up 18:26:21 that seems non-controversial and not-that-hard to do 18:26:37 (others agree?) 18:26:42 agree. 18:26:59 kk. so strike the idea of next android release being alpha on top of v99 18:27:04 👍 18:27:14 that gets the next slot (whenever that winds up being) 18:27:32 but regardless of what is being released when i have a question about the general timing of "interleaving" 18:27:38 on the bi-weekly cadence 18:27:54 do people have the calendar in front of them by any chance? 18:28:05 I have 18:28:09 if you noticed for example the 11.5a6 Dekstop Sign/Tag is on 3/9 18:28:17 (after rebasing that starts today) 18:28:19 isn't alternating an ideal scenario, but not strictly needed until Android catches up? so, break the alternating schedule for the next ~1 month and get back in sync with Mozilla? 18:28:29 the release based on that sign/tag doesn't happen until 3/15 18:28:33 oh no! 18:28:37 what happened to richard? 18:28:44 richard 18:28:49 yo 18:28:51 richard's okay 18:28:55 #rurallyfe 18:29:07 lolol 18:29:33 did you miss some scrollback? 18:29:39 (i can repaste) 18:29:40 yeah I think so 18:29:41 +aguestuser | if you noticed for example the 11.5a6 Dekstop Sign/Tag is on 3/9 18:30:01 [+aguestuser | (after rebasing that starts today) 18:30:10 +aguestuser | the release based on that sign/tag doesn't happen until 3/15 18:30:16 (and we're back) 18:30:22 so my question is: 18:30:32 i am also rebasing this week 18:31:01 and would like to release shortly after signing/tagging 18:31:15 but assuming i sign/tag at the end of the week 18:31:23 i can't ship until after the desktop release goes out 18:31:38 (if for no other reason because the dekstop has "taken a lock" on the release number) 18:31:56 this makes it tricky to neatly interleave the releases 18:32:07 in our ideal-case scenario 18:32:18 (once we're caught up and the android releases don't take forever) 18:32:26 so i guess i'm curious: 18:32:48 (1) is there a reason we delay for a week btw/ signing/tagging on desktop and releasing? (should we do that on android also?) 18:33:22 so the delay is a builtin safeguard in case shit breaks 18:33:24 (2) if we *don't* want to observe that delay on android how to handle the fact that this lag inserts an artificial delay of ~1 week in releasing android once it's essentially ready? 18:34:10 in theory there's no reason to delay if everything is fine, but i'd rather have that safe window in case we need a build2 or a build3 so we aren't making people wait based on the schedule 18:34:14 oh also: i *think* waiting on that lag also means we have to delay signing/tagging android releases until after dekstop release went out 18:35:03 hmm yes 18:35:14 it's not strictly needed, but it is cleaner if you wait 18:35:32 technically speaking you can sign/tag once we've verified matching desktop builds 18:35:58 okay, so we could just follow the same lag? 18:36:02 at that point we shouldn't need to touch the tor-browser-build branches for that build 18:36:30 thus far we've always shipped every android build the same day(ish) as signing/tagging 18:36:52 same day as binary signing you mean? 18:36:53 but allowed for safety by leaving new branch in nightly for a bit before signing/tagging 18:37:00 sorry i meant signing/tagging to mean signing/tagging git commits 18:37:09 1 week is a longer delay then we've used in the past - we usually put 2-3 days, and then update the changelog with subsequent buildN's, if needed 18:38:12 richard: we can release an android release in ~24 hours, from tagging to publishing 18:38:15 what is the difference btw/ (1) the buffer we leave between rebasing patches onto new upstream branch and merging that (shipping it to nightly) and (2) the buffer we leave between signing/tagging commits and publishing release? 18:38:37 i had thought that (1) was the thing we did to catch bugs, etc. 18:38:50 is (2) just extending (1)? (leaving the same code in nightly for longer?) 18:39:16 since these delays are holding something from going from nightly to alpha, right? (2) is just doing more of that? 18:40:10 in any case: i don't necessarily need to know the motivation to figure out the scheduling stuff. 18:40:32 the fact is just that interleaving gets a bit tricky b/c of different lenghts of delay between sign/tag and release on different platforms 18:40:49 i had tried to puzzle it out on my own last week on calendar but figured it would be good to check in with others! 18:40:53 the (2) buffer in my mind was just to make the release process less crunchy from a timing perspective (and to take into account the long iteration time involved with building releases) 18:41:03 akc 18:41:07 (on the desktop side) 18:41:12 yup yup! 18:41:26 if android is easier/faster for whatever reason then by all means do what makes sense for you 18:41:40 okay, but: what about issue numbering 18:41:46 let's say we start this week 18:41:57 and you are building 11.5a7 (desktop only) 18:42:05 i am building 11.5a88 (android only) 18:42:08 there isn't a technical reason we can't release 11.5a7 before 11.5a6 18:42:19 this gets to the root of my question 18:42:34 right now i am trying to take later numbers (b/c i'm assuming i will take longer while still learning) 18:42:42 but if eventually android finishes faster 18:42:52 i had thought it should maybe have the first release number 18:43:01 maybe it no longer makes sense using the same version numbering system 18:43:04 * donuts quickly exits 18:43:11 donuts: my thought as well 18:43:15 donuts: I was wondering the same 18:43:25 can we use some adaptation of fenix's for android? 18:43:32 (if, e.g., android 11.5a8 is ready to ship by thursday, but desktop is not ready to ship until next tue) 18:43:34 eventually, stable android should be build with desktop, and alpha should be at the ~2 week mark which should not align (or block) a desktop-only release 18:44:06 but it seems like desktop starts to do an alpha-related rebase at the top of every month 18:44:12 which android is also doing 18:44:28 (since that's when the mozilla releases roll over for both of us) 18:44:31 but desktop releases after ~1 week, and android builds/releases after ~2 weeks 18:44:50 i see. okay. makes sense 18:44:52 but if android is ready to release after ~1 week 18:44:58 we still want to hold until ~2 weeks to publish? 18:45:05 we should stay with the 2 week cadence 18:45:17 sysrqb: in the current calendar 18:45:20 you can consider expiditing, but that's on a case-by-case basis 18:45:27 dekstop does *not* release at ~1 week mark 18:45:35 it releases at ~2 week mark 18:45:42 err 1.5 18:46:04 still seems okay :) 18:46:08 but regardless... i think i understand the desired cadence 18:46:18 (1) moz rolls over 18:46:19 (2) everyone starts rebasing 18:46:25 (3) dekstop releases alpha 18:46:29 (4) android releases alpha 18:46:43 (5) everyone releases stable together at same time? 18:47:00 (with shared version number for stable release?) 18:47:11 (3') desktop/android stable 18:47:16 (4) desktop alpha 18:47:22 (5) android alpha 18:47:36 stable release almost always comes first 18:47:53 hmm... not in the calendar i worked off last week 18:48:07 or rather it was step (0) 18:48:10 because we want to get security updates out to our users asap after mozilla releases their version 18:48:13 happened before moz rolled over? 18:48:43 ah, i see 18:48:46 i think. 18:48:57 there are multiple versions in play 18:49:06 let's look at this month 18:49:14 or the stable desktop release happens more or less concurrently w/ (1) 18:49:29 (the day that moz rolls over and the day after we started rebasing on the next alpha) 18:49:46 yes, this month! 18:50:09 (which might be tricky b/c security hotfix, but yes: a concrete example would be great) 18:50:33 11.0.7 stable desktop was calendared for release tomorrow 3/8 18:50:47 mmhm 18:50:55 (on top of esr91.7) 18:51:16 (let me know when/where I should jump in) 18:51:50 i am actually now confused by the fact that 3/8 says it is a stable release on esr91.7 but we also start rebasing alpha onto esr91.7 today 18:52:10 correct :) 18:52:39 (again the 1 week lag is just built-in buffer, really I ttry to treat these dates as due dates, not start dates, with limited success) 18:53:08 "+sysrqb | correct :)" <- that is not how we do it in android? 18:53:18 fenix versions land in alpha beforfe stable 18:53:27 i mean jump in at any point 18:53:32 i'm actually sort of hopelessly lost now 18:53:40 :) 18:53:43 it's all good 18:53:52 but i am not sure the desktop versioning question affects my question 18:54:15 let's actually look at next month, under the assumpt Android successfully jumps onto the FF99 train 18:54:51 by next month we mean april? 18:54:59 yes 18:55:01 https://nc.torproject.net/apps/calendar/dayGridMonth/2022-04-01 18:55:21 on 4/4 moz rolls over and we start rebasing 18:55:21 shortly after 2022-04-04 Mozilla will tag 91.8esr and 99.0 18:55:27 correct. 18:55:35 we start with stable 18:55:46 for the past month, we've been testing 99beta in our alpha release 18:55:53 (~2 weeks, really) 18:56:08 we now rebase our patches onto the 99.0 stable release branch 18:56:20 and desktop patches are rebased onto 91.8esr branch 18:56:39 we build these new branches and release a stable version on 2022-04-05 18:56:49 okay, so without need for rebasing, we quickly publish stable targetting both 91.8esr and 99 18:56:59 on 04-05 18:57:05 following 18:57:06 (hrm. okay, the tag date was wrong, it should be around 1 week earlier) 18:57:10 (and had completely missed that in my release scheduling) 18:57:44 also if looking at this calendar, note: that i've got fnx99 where i should have fnx100 18:57:58 sorry, we do rebase the android patches from the 99 beta branches to the stable branches 18:58:14 onto maint-11.0 18:58:18 (in this case) 18:58:30 yes 18:58:49 and then we rebase fnx100 onto new (soon-to-be-alpha-release) branch 18:59:03 correct 18:59:14 and that will take around 1-2 weeks, for rebasing and toolchain updates 18:59:29 after 2 weeks, we should be ready for that alpha release, on schedule. 18:59:41 desktop, goes first, then android 18:59:57 and if helpful for android to wait after signing/tagging to publish we can 19:00:16 (i guess a helpful insight for me from this is there is no blocker to signing/tagging as soon as i want) 19:00:40 and: it's still aspirational that any of the rebasing for alpha branches will happen "fast" (under any definition of "fast") 19:00:46 correct, and we should probably release a combined desktop/android alpha release based on 91.8esr and Fenix98 after the 1 week delay 19:01:00 because there isn't really a reason not to update android at the same time 19:01:28 mhnmm 19:01:30 err... fenix100? 19:01:44 (if we're talking april?) 19:02:21 but where did the idea of interleaved releases go then? 19:02:23 err, no, Fenix 99, sorry. we release desktop/android stable based on 91.8esr and Fenix99, and then after a week I propose we release the alpha for both desktop and android 19:02:49 and then the android-only alpha based on Fenix100 comes along at the ~2 week mark 19:03:40 but this doesn't need to decide right now 19:03:46 we can table this until the next meeting 19:03:48 "then after a week I propose we release the alpha for both desktop and android" <-- gosh. currently alpha releases are "ahead." ie: alpha is on fenix v96, while stable is on v94 19:03:53 is that an anomaly? 19:04:17 i was trying to map out next 2 months assuming that stable is always 1 version behind alpha in terms of fenix version 19:04:32 also: we are at 5 minuts past time 19:04:38 sorry for swallowing entire meeting 19:04:38 *35 19:04:55 release meeting is half an hour :) 19:05:00 oh really? 19:05:00 lol 19:05:02 geeze 19:05:03 aguestuser: no, that is how it should work: alpha -> 99beta, stable -> 99, alpha -> 99, alpha -> 100beta, stable -> 100, etc. 19:05:11 ha yes technically PieroV is right but that's totally fine, seems like there's nothing else following this 19:05:27 true 19:05:38 ok aguestuser, are we good now w/ regards to android release schudling? 19:05:43 lol 19:05:47 :p 19:06:12 i am reading the above tyring to parse the differentce between 99beta and 99 19:06:17 etc 19:06:29 aguestuser: beta release verses stable release 19:06:41 mid-month verses beginnning-of-next month 19:06:55 https://wiki.mozilla.org/Release_Management/Calendar 19:07:21 ok, right so if we track that we presereve the invariant that alpha is always a "branch" ahead of stable 19:07:23 but not a version 19:07:27 99beta is mid march, and then 99 is beginning of april 19:07:28 (which is the mental model i had) 19:07:44 or not a major version 19:07:49 yes 19:08:16 okay, i think i need to take somet time to stare at calendars again before i can actually project 2 months out 19:08:18 i think i threw a wrench into you mental model when I proposed releasing android-alpha based on 99 19:08:24 but that's okay 19:08:47 i will start with figuring out what version numbers to give our android-only emergency releases 19:09:14 then figure out what to number the next android stable (v96) and android alpha (v99) releases. 19:09:41 with an honest effort to try to parse whetner i mean v99beta or just v99 19:09:48 and coordinate with richard on what our release numbers should be 19:10:10 (punting to next time on question of doing new version number scheme should be! yes?) 19:10:20 /s/should/might 19:10:30 (this is why we haven't successfully scheduled future version-number release dates, only version-less release dates) 19:12:05 ok. i am officially done wasting other people's time in public. :) 19:12:13 donuts: you may need to end the meeting :) 19:12:17 nope 19:12:19 haha yes will do 19:12:21 I have still two points 19:12:29 onion auth is fixed, tor-browser!261 19:12:30 lol gogogo 19:12:32 please go ahead pierov 19:12:34 nooooooooooo 19:12:43 are we going to do a .7.1? 19:12:48 fantastic! what was the issue? 19:12:51 dope 19:12:55 as Richard said, a missing await 19:13:08 lol called it 19:13:10 * richard dabs 19:13:15 lol 19:13:30 or are we waiting for next release? 19:13:43 (sorry: my "noooo" was at meeting ending! not at PieroV saying stuff. really excited for someone else to say stuff! :)) 19:13:49 hahaha 19:13:56 and in case I would also slip the IPv6 circuit display 19:13:57 we could do a simultaneous android/desktop stable release with the fix in 19:14:18 okay, then IPv6 can wait 19:14:27 yeah IPV6 can wait regardless 19:14:46 PieroV: what's the summary on the onion auth issue? 19:14:51 I proposed it in the backport asap small features spirit 19:15:00 is it just bad UX if the bug is hit 19:15:03 tor-browser#40802 19:15:04 or does it just not work at all 19:15:21 it doesn't work at all 19:15:30 ugh damn 19:15:36 does that mean there should be a fix for it next week (in x.x.7.1)? 19:15:54 ok let's plan on new stable+alpha desktop releases next week then 19:16:31 i'll get to merging and tagging and whatnot this week 19:16:38 okay, works for me 19:16:53 aguestuser: i'll follow up wiht you later abotu the right version number to use :3 19:17:24 no more points from me :) 19:17:40 richard: ack :) 19:18:01 okay I'm gonna press the button 19:18:03 ok 19:18:04 3... 19:18:06 2... 19:18:07 1... 19:18:10 #endmeeting