16:59:09 #startmeeting Network team meeting: 3 April 2018 16:59:09 Meeting started Tue Apr 3 16:59:09 2018 UTC. The chair is nickm. Information about MeetBot at http://wiki.debian.org/MeetBot. 16:59:09 Useful Commands: #action #agreed #help #info #idea #link #topic. 16:59:16 hi 16:59:16 hi 16:59:24 hello world 16:59:28 hi 16:59:38 hi everybody! the meeting pad for the week is at https://pad.riseup.net/p/XkE251jt768D 16:59:57 ! 16:59:58 hey 17:01:00 hi 17:01:17 isabela: would you like to get us started? 17:01:26 yes 17:01:30 hi all ! 17:01:38 roadmap check time folks :) 17:02:09 i feel we are over our capacity for all things we want to be done by 0334 freeze (may 15) 17:02:35 especially given as how we're still on "fix must-fix bugs in 0.3.3." :) 17:02:35 * dmr lurks but is currently in discussion about a paper; will catch up on scrollback later 17:02:40 if there is anything that is not time sensitive and can be moved that would be a good time for it 17:02:50 * asn looks at his 034 stuff 17:03:09 all things placed at our roadmap for april are things we are considering for 034 17:03:38 plus whatever might be on 034 queue that is not sponsor work 17:03:41 I think for now, on my part, I'm confident that May 15th will be enough for the tasks, I'll know more in a week since I'm started this week Roadmap stuff that I postponed due to 033-must 17:03:46 I wonder if we could have a points column there? some of these items are much bigger/smaller than others 17:03:49 mikeperry: do you think we could/should move some of the vanguard/2-guard stuff? 17:04:03 I think I can get my stuff done on time, barring more surprises than usual 17:04:06 nickm: good point 17:04:14 the two tasks i have can easily be moved forward. they are not directly depending on little-t-tor 17:04:44 how do we do this? we just move our tasks or do you want to be handling all that, isabela? 17:04:46 i am not sure if folks prefer to do it now at this meeting or off meeting 17:04:51 I think "Reduce CPU usage when idle" will leave us with some parts not finished, but some parts done. 17:05:04 asn: I am not scheduled to do anything else exceopt #25511, which someone else just voliunteered for and I also feel is less important 17:05:05 dgoulet: we are also supposed to add #25552 for 034... 17:05:06 you can move rows around at google spreasheet 17:05:12 ok 17:05:25 asn: yes 17:05:31 mikeperry: I also see you on lots of sponsor V stuff 17:05:34 right now they are group by sponsor cuz of the 'visuals' hehe 17:05:35 o/ 17:05:52 mikeperry: well, there is the vanguard tor patches, and the 2-guard stuff, (and the rest of the vanguard stuff that are not tied to 034) 17:06:03 isabela: what do you think about marking stuff that we can merge in a partially completed state? 17:06:04 i think a lot of the sponsor3 stuff is general improvement, so we can continue to work on them after 034 17:06:28 For example, #25500 and #24986 are useful even if we only do some of it in 0.3.4 17:06:50 nickm: yes, that is all of the vangurd/2-guard stuff. as I said, I don't really have anything else 17:07:18 I would say I am happy to chill, but that is not really the case either ;) 17:07:24 nickm: lets bring forward to the roadmap the one we are doing on each release then 17:07:28 mikeperry: ack :) 17:07:34 nickm: paste the child ticket there too 17:07:52 dgoulet: how do you think we should fit in the rev counter issue? 17:07:56 isabela: I don't understand what you're asking for? 17:08:12 hi all 17:08:31 dgoulet: just add #25552 with R+, and hope to find time to do it? 17:08:32 asn: I think for 034 is fine to the cost of moving things to 035 on my plate at least 17:08:35 nickm: sorry i thought you were saying these were child tickets of an item at the roadmap 17:08:43 nickm: leave in april whatever will go to 034 17:08:53 nickm: what is not going to 034 move it to may 17:09:18 you should be able to carry the row around and place it wherever you want at gdocs 17:09:32 dgoulet: aight im adding a row for #25552 17:10:01 isabela: I was asking about tickets where I think we will do some of it, but not all of it. 17:10:31 you can copy them in may as well 17:10:43 so we know the final release target wont be 034 17:11:05 But some _will_ be in 034 17:11:10 yes 17:11:12 for instance, #25500 17:11:13 not final right 17:11:17 just part of it 17:11:23 some of its children will go in 034 17:11:25 some might wait 17:12:04 having it listed in both months seems plausible. if that's confusing, maybe add a letter to the first one indicating that it's "C"ontinued below? 17:12:11 yes, so if you want to you can always have the children there to reflect all of them, but is fine if you just repeat the master ticket on both months too 17:13:11 ok, tried to do that 17:13:33 I think that will give the groups I'm in enough space to definitely get all the other stuff done for the May 15 freeze. 17:13:45 I hope 17:13:53 cool 17:13:59 nickm: is looking good tx 17:14:41 are we good with the roadmap for now? Should we talk about 033? 17:14:44 tbb does it that way too, when they will work on something with child tasks that carries on over months, they just copy it on the next month 17:14:57 so that will help my brain 17:15:07 when i look on different roadmaps i read the same thing 17:15:33 nickm: yes good to move (/me thinks) 17:16:15 we have 30 tickets in the 0.3.3.x milestone. 19 of those are marked 033-included. 17:16:45 let's start by looking at the 11 that aren't: 17:16:45 https://trac.torproject.org/projects/tor/query?status=accepted&status=assigned&status=merge_ready&status=needs_information&status=needs_review&status=needs_revision&status=new&status=reopened&keywords=!~033-included&milestone=Tor%3A+0.3.3.x-final&group=status&col=id&col=summary&col=owner&col=type&col=priority&col=component&col=version&col=reviewer&order=priority 17:17:02 should any of these be 033-must, or included in 033 for some other reason? 17:17:20 I'm going to put my needs_review tickets in since both are fast-fix 17:18:01 that leaves 9 not-included tickets: 6 are needs_information, and 3 are new. 17:18:23 should any of those be things that we do in 0.3.3? 17:18:32 none of the HS stuff is critical for 033 IMO 17:18:39 or the guard stuff. on the list of 11. 17:19:14 asn: Mark those as 033-removed-20180403, and I'll take them out of the milestone later? 17:19:21 ok 17:20:09 yeah #25692 seems to be 034 regression so far so we can remove it from 033 17:20:30 dgoulet: it has been observed in 033, though. But we can safely call it 034-must regression, since it happens much more in 034 17:20:36 doing that now 17:20:57 oh huh... so we might have released it in 0.3.3.4 then 17:21:41 we're left with #24857 17:22:07 "milestone: Tor: 0.3.3.x-final => website redesign" hmmm lol 17:22:12 I,ll fix that ^ 17:22:19 thanks 17:22:34 we're left with only needs_info tickets. Great! 17:22:45 Let's turn our attention to the tickets that _are_ triaged in: 17:22:52 https://trac.torproject.org/projects/tor/query?status=accepted&status=assigned&status=merge_ready&status=needs_information&status=needs_review&status=needs_revision&status=new&status=reopened&keywords=~033-included&milestone=Tor%3A+0.3.3.x-final&group=status&col=id&col=summary&col=owner&col=type&col=priority&col=component&col=version&col=reviewer&order=priority 17:23:29 dgoulet: ack on #25616. will take a look this week. 17:23:36 and by this week i mean tomorrow. 17:23:43 dgoulet, ahf, isis, catalyst, asn: you have tickets in accepted or assigned for 0.3.3! 17:24:05 yep! i have one left that i'm gonna look at tomorrow was the plan 17:24:06 on my part, they _need_ to go in 033 17:24:07 there are 6 in needs_review; all but #24782 have reviewers. 17:24:23 isis, asn, dgoulet: you have tickets for 033 in needs_revision. 17:24:24 i *just* marked that as needs_review before the meeting 17:24:30 * dgoulet grabs #24782 to review 17:24:49 Does anybody think that they're not going to get all these tickets done this week? 17:24:52 i'm assigned #24544. it's an hsv3 spec patch for 033. i know it's important, but i'd like to do it with a bit less pressure. 17:25:09 i'll likely do it this week or the next one, but can i put it in 034, so that i don't have it as so urgent? 17:25:22 #25347 is a much more important task for 033 for me. 17:25:27 Since it's a spec patch, label it post-stable and we don't have to call it release-blocking 17:25:40 * ahf plans on getting #25245 done tomorrow 17:26:12 nickm: how do i label it post-stable? should i just put it as a keyword? 17:26:16 yes please 17:26:18 ok 17:26:41 odne 17:26:50 I'm closing #25316 as a #24700 duplicate. 17:26:56 nickm: i count only one 033 ticket assigned to me, #25061 17:26:56 nickm: great 17:27:18 catalyst: that looks right. You are also listed as reviewer for #25679 17:27:35 nickm: ack 17:27:46 guys #25347 is pretty important! it ideally needs more feedback from a subset of roger, mike and nick. 17:28:13 it's conceivable that it doesn't get in 033 and it's not gonna be the end of the world. but more analysis would be useful! 17:28:28 i plan to work on it this week fwiw. 17:28:44 asn: Is there anything we can do on it where we know that doing so would help? 17:28:48 oh, i have #25208 17:29:21 nickm: the thing we previously identified as actionable seems like it would also do harm. 17:29:25 nickm: we should think some more. 17:29:33 should i count descriptor uploads in the dirauth state file, or the client state file? 17:29:40 s/client/OR/ 17:31:08 mikeperry: i think you should finish up #25581. it's pretty close to the end, but it just needs your final decision on what you want vanguards to be for now. 17:31:36 arma5: any insight on isis's question about #25208 ? Fwict this might not even need a state file change; just a rate-limit. It would be okay if the rate-limit were reset on server restart. 17:31:39 mikeperry: personally, i would go with no underscores and a fat warning on the man page. but i dont have strong opinions. 17:31:54 asn: ack will look. that is my preference too, I think 17:32:15 great 17:32:26 nickm: agreed, no need to write anything to disk. i think the rate limiting should be on the dir auth side, because relays can't be trusted to obey it, that's the whole point. but also, i think this topic is tied into ...whatever that mystery bug ticket is. 17:32:32 asn: I'm adding "read and understand #25347" to my todo list, sadly. 17:32:46 nickm: #25686 17:33:09 nickm: i think perhaps mike and roger could also deal with it. you would be useful because of prop271 knowledg. 17:33:15 nickm: i think simply rate limiting descriptor uploads, full stop, is probably the wrong approach. we need to look at the descriptors and be smarter than that. 17:33:15 do we have a system for rate limits somewhere? (what function(s) should i look at?) 17:34:04 isis, arma5: with this in mind, I suggest that we defer #25208. 17:34:04 does anyone have code to pick specific RPs when connecting to a hidden service? I want to test for regressions to #14917 in the vanguard code 17:34:18 It's a regression, but we clearly have no idea what to do about the underlying problem. 17:34:40 mikeperry: I think I might but it was about pinning an RP so I could do test on my relay, ping me about it if this is interseting to you 17:34:42 mikeperry: yes there is such a thing. tor2webrendezvouspoints or whatever it's called. 17:34:47 i'm fine with defering #25208 17:34:56 mikeperry: Tor2webRendezvousPoints 17:35:03 okay, any objection to deferring 25208? 17:35:34 asn, mikeperry: won't work without tor compiled with Tor2web enabled though 17:36:22 I guess if #25208 is not putting too much pressure on dirauth, could be OK to defer but not loose track of 17:36:24 ah nice is a routerset, so I can use to test /16 restrictions too 17:36:58 defering 25208 sounds fine to me. i think 25686 is more urgent than 25208, since 25686 is actually "find the bug", rather than "put a bandaid without understanding what's going on" 17:37:03 arma5: isis and I are apparently in disagreement about what you said we should do for rate-limits. I think you said "rate limits are a bad idea" here 17:37:15 #25208 deferred. 17:37:46 oh, i don't think rate-limits are a bad idea, i just think enforcing anything OR-side is a bad idea 17:37:46 * nickm has blessed #25686 with a few 034 categories 17:37:59 isis: I think roger thinks they're a bad idea. 17:38:06 mikeperry: what about the guardfraction thing? do you think we should act on it in a 033 timescale? or just let it for 034, since we have already warned the dirauths to shut the script down? 17:38:17 nickm: aha, i see 17:38:30 i think rate limiting uploaded descriptors full stop is wrong. i think rate limiting certain kinds of descriptors could be a fine idea. what kinds exactly? it depends why we're being flooded. 17:38:53 arma5: I think I want to close #3490 as "not a bug" 17:38:53 as in "just fix the underlying bug instead of forcing good behaviours"? 17:39:00 err 17:39:03 asn: if no one is running it, then it does not seem urgent I guess 17:39:05 #3940 17:39:15 mikeperry: ok 17:39:24 nickm: yes, i've been unable to reproduce #3940 17:39:28 i will mark it for 034, and make sure that i notify the dirauths some more 17:40:22 Is there a reasonable thing we can do for #21394 ? 17:40:29 mikeperry: should #21394 still be in needs_information? 17:42:17 oh yeah I can make a git branch and set to merge ready 17:42:25 mikeperry: that would be a good thing 17:42:42 okay, we've progressed on 033 17:42:50 and we have 17 minutes left in the meeting time! 17:43:34 everybody, please look over others' updates and ask them questions in #tor-dev after this meeting -- we probably won't get through the discussion list 17:43:50 next topic is rotations! 17:43:58 isis: I hope I have implicitly answered your question about where the code is by making GitHub send you an email 17:44:36 asn, catalyst, dgoulet, isis: you are on rotations this week as bug triage, "community hero" (to be renamed), coverity, and CI respectively! 17:44:40 https://trac.torproject.org/projects/tor/wiki/org/teams/NetworkTeam/TeamRotations 17:44:48 ack 17:45:11 (fwiw i noted how my week went as a community hero on my status update on the pad) 17:45:18 (can the ppl on these positions last week sync with those on the same position this week?) 17:45:22 i'm CI, yeah! everyone lmk if your CI is borked and it wasn't (or was) your doing 17:45:26 asn: great 17:45:38 i think some knowledge passing will help folks 17:45:46 asn: awesome! That's great process 17:45:53 * asn still hasn't looked at status updates 17:45:57 "community guide" seems nifty 17:45:59 it looks like the CI was doing fine yesterday 17:46:16 (does the CI role include jankins?) 17:46:23 err, jenkins, heh 17:46:24 undefined! 17:46:52 isis: i think we want to take GitHub as primary, and handle others when resources allow 17:46:55 some jenkins builders have been sad; making them happier if real problems exist might be cool, but it's less urgent than the other stuff. 17:47:00 +1 catalyst 17:47:13 jenkins does seem a bit janky sometimes 17:47:17 asn: nice list for the community hero work 17:47:18 cool 17:47:44 i personally find jenkins too fragile to be of use; maybe others get more utility from it 17:47:47 oh, another thing: I'm going to be sending out another "help me triage" email like I did for 033, but this time for 034. When everybody's done with that, I'll remove tickets from 034. 17:48:13 How long should I give everybody to help mark stuff that should _not_ get removed? 17:48:31 datapoint: Jenkins builds on all Debian stable release which is _important_ for our packages so we can't just ignore it 17:49:04 not fixing those ^ will delay packages and make our Debian packager potentially unhappy 17:49:15 well, many of the failures are just plain not ours to fix: 17:49:17 eg https://jenkins.torproject.org/job/tor-ci-linux-0.2.9/ARCHITECTURE=amd64,SUITE=stretch/199/console 17:49:54 dgoulet: is there a way to make a jenkins dashboard thingy for "stuff that's actually a high priority for us to fix"? 17:50:14 nickm: woa... Java so this is Jenkins issue itself... meh 17:50:55 contrast with https://jenkins.torproject.org/job/tor-ci-linux-master-expensive-hardening/1781/ARCHITECTURE=i386,SUITE=buster/console -- something is going wrong with all the ASAN executables there, which suggests a debian or an installation problem to me, but I have no idea. 17:51:10 * nickm asks earlier question again: 17:51:17 How long should I give everybody to help mark stuff that should _not_ get removed from 0.3.4? 17:51:17 these should be looked at imo: https://jenkins.torproject.org/view/packages%20-%20debian/ 17:51:30 nickm: 3 days? 17:51:31 i feel like with jenkins i spend a really long time digging through logs only to find that either 1) there's no bug (at least that we could/should fix) or 2) something just randomly failed for jank reasons 17:51:48 isis: +1 17:52:11 but mostly that the amount of time i spend is not really productive, like 2 minutes of looking at travis output tells me if it was spurious and/or what the bug is 17:52:14 is this the fault of jenkins itself, or something wrong with our jenkins install / setup? 17:52:16 that is the current situation more and more but we should inform the maintainer of Jenkins 17:52:24 I think it's worth checking jenkins for regressions periodically 17:52:40 and with jenkins it's like 30 minutes before i feel like i understand enough to report something 17:52:53 what does travis output that jenkins doesn't? 17:53:06 just more clicks to get to the console output 17:53:24 is that the issue? clicking to the output? 17:53:38 (5 minutes) 17:53:54 jenkins needs more clicks, the links are harder to navigate, there are awkward hierarchies that aren't obviously documented, etc. 17:53:56 Does everybody think that next monday is a reasonable deadline for marking 034 tickets to retain? 17:54:36 there's a lot of jenkins builds, no easy way for me to know what setup scripts/configs they are using, no clear way to just show me the thing that failed, no clear way to show me if multiple build failures were related (e.g. "everything on 0.2.9 on every arch is failing" etc) 17:54:45 nickm: I would go shorter because going over the list doesn't take long tbh... but I guess 7 days is good as we would go meeting to meeting 17:55:49 I'm sure we all agree on Jenkins being more complicated and less good than Travis ... it is just that currently Travis doesn't replaced Jenkins for our Debian stable packages which are _important_. (Unless travis does that and then I withdraw my case :) 17:56:28 or for windows 17:56:32 or for other stuff 17:56:32 dgoulet: we might be able to get that kind of coverage using GitLab CI, which would be easier to troubleshoot 17:56:54 (3 minutes left. we won't resolve all CI issues right now) 17:56:57 fair enough, but in the meantime, we need to look at Jenkins, that is all I'm saying 17:57:16 until we have the better alternatives, the best thing we have is still jenkins :/ 17:57:49 one last note on rotations -- if there are big issues under your rotation, you're not required to tackle them all yourself. Getting help is good! 17:58:00 let's move CI discussion to #tor-dev post-meeting 17:58:13 :-) 17:59:18 juga: how come u not sure about sbws and bwscanner? 17:59:29 seems like sbws is taking shape pretty quickly (and also invisibly) 17:59:40 asn: yup 17:59:45 hopefully in a month pastly will hear from his employer that it is freeee 17:59:52 yep also hoping so 17:59:53 and then it will be less invisible 18:00:07 juga: you have access in the codebase? 18:00:09 the idea initially was to work on bwscanner, but looking sbws i think is worther 18:00:13 asn: yes 18:00:15 Yeah I think the invisible part is creating reservations, which is fair 18:00:18 juga: you like it? 18:00:20 pastly: one permission, once, is good foreverish, i hope? 18:00:35 asn: yes :) 18:00:40 (all development after that will just be minor tweaks after all) 18:00:42 okay, we're out of time. 18:00:44 good to know 18:00:45 thanks, everybody! 18:00:49 #endmeeting