15:01:01 <karsten> #startmeeting metrics team meeting
15:01:01 <MeetBot> Meeting started Thu May 21 15:01:01 2020 UTC.  The chair is karsten. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:01:01 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
15:01:04 <karsten> hi!
15:01:17 * karsten put a few topics on the agenda pad...
15:02:08 <karsten> Public pad URL is: https://pad.riseup.net/p/tor-metricsteam-2020.1-keep
15:02:25 <karsten> please add more topics, and feel free to put them at the top.
15:02:39 <gaba> hi!
15:03:11 <karsten> so much to talk about.
15:03:17 <dennis_jackson> I had a day off today for a public holiday and spent a bit of time on #34257
15:03:25 <karsten> oh!
15:03:37 <dennis_jackson> In the process of writing it up (just got my Trac permissions sorted)
15:03:46 <karsten> neat!
15:04:12 <karsten> added a note to the pad that we should talk about that.
15:04:32 <acute> I was about to update #30586
15:05:08 <karsten> added a note.
15:05:19 <karsten> maybe let's start.
15:05:30 <karsten> we might not have enough time to cover all topics anyway.
15:05:36 <karsten> Update OnionPerf to TGen 1.0.0 (#33974)
15:05:51 <karsten> hi jnewsome!
15:05:59 <jnewsome> karsten: hello!
15:06:07 <karsten> I was just about to talk about #33974.
15:06:21 <karsten> where my main question was: do you need any help here?
15:06:44 <karsten> it's not blocking anything, I'm just trying to unblock you if that is the case.
15:06:58 <jnewsome> looking up the ticket...
15:07:06 <jnewsome> but I assume it's "migrate to tgen 1"?
15:07:09 <karsten> yep.
15:07:46 <jnewsome> main thing that might be helpful is an environment to run in? I think you mentioned getting access to VM(s)?
15:07:56 <karsten> yes.
15:08:00 <karsten> I'll email you about that.
15:08:27 <jnewsome> other than that, I haven't had much time to work on it yet and am mostly learning how things work. I was having some trouble getting the demo working
15:08:46 <karsten> that's fine.
15:08:46 <jnewsome> probably going to poke at it some more today and/or tomorrow
15:09:17 <karsten> okay, great, will email you about getting access to a testing VM in the same place as our other instances are running.
15:09:26 <jnewsome> sounds good. thanks!
15:09:29 <karsten> cool!
15:09:31 <karsten> moving on:
15:09:32 <karsten> requirements are not included in setup.py (#30586)
15:09:41 <karsten> acute: you have an update?
15:10:13 <karsten> my main question there was that we might want to avoid installing packages automatically using pip.
15:10:17 <acute> yes, I was wondering whether there is a distinction here that we need to make - how do {researchers, developers} install and use onionperf on their machines vs how we deploy onionperf in Tor
15:10:38 <karsten> that's a fine question!
15:10:42 <acute> For general purpose use/development, using python-setuptools (which is a standard way of packaging python modules and uses pip to manage dependencies) and then running onionperf on the command line is reasonable, imo
15:10:56 <acute> for 'official' deployments, not so much
15:11:31 <phw> how did we set up onionperf on our deployed instance?
15:11:35 <acute> there, it makes more to install dependencies directly from Debian, and use a systemd service to run Onionperf
15:11:55 <karsten> or, a tmux session...
15:11:57 <acute> this is what the ansible workflow does
15:12:20 <karsten> so, what does that mean to the patch on the ticket?
15:12:26 <jnewsome> I see in the ticket we have a general policy of avoiding such package managers; why is that? (and is it worth maintaining 2 ways of doing it?)
15:12:36 <acute> tmux could work, but it means things get more manual
15:12:40 <acute> and are harder to monitor
15:12:56 <acute> the actual 'python3 setup.py install' step is not even necessary, as you can just run onionperf directly from the git repository
15:12:58 <karsten> acute: yep. it's just the way it is right now. but it totally doesn't have to be that way.
15:13:15 <acute> as long as your PYTHONPATH is set
15:13:24 <karsten> we have avoided using package managers in java code bases.
15:13:33 <karsten> I don't know what the general tor policy is here.
15:14:09 <karsten> and again, I'm not a python person. if this is the way to do it in python, I'm fine with that.
15:14:17 <karsten> I mostly wanted to flag this as potential issue.
15:15:01 <karsten> acute: do you want to write an update on the ticket?
15:15:06 <karsten> with suggestions?
15:15:48 <acute> yes. I think in the long run we should get onionperf in debian
15:15:54 <phw> for what it's worth, bridgedb (also python) uses pip in deployment but our sysadmins would prefer us to use debian packages instead
15:16:07 <karsten> phw: interesting point.
15:16:18 <jnewsome> phw: do you know why they prefer that/
15:16:40 <karsten> acute: right now it changes too fast to be a useful package in debian, I think.
15:16:47 <phw> jnewsome: you then don't have to worry about vulnerabilities in libraries because debian takes care of that for you
15:17:11 <jnewsome> my understandig is that it's pretty standard to use pip when dealing with a source repo,a nd only use the system package manager if you've packaged it up
15:17:14 <jnewsome> phw: ah fair point
15:17:50 <karsten> okay, let's take this back to the ticket.
15:17:51 <acute> yes, agreed - this is a more of a long-term idea
15:18:16 <karsten> I put a note on the pad that you're writing an update on the ticket.
15:18:58 <karsten> okay, moving on.
15:19:02 <karsten> Analyze unusual distribution of time to extend to first hop in circuit (#34257)
15:19:08 <karsten> dennis_jackson: you had something here?
15:19:24 <dennis_jackson> Yes, quick summary
15:19:44 <dennis_jackson> https://raw.githubusercontent.com/galadran/onionperf-guard-analysis/master/images/histograms/bt1_op-hk2.png
15:19:58 <dennis_jackson> The time to build the first hop for hk has discrete bands
15:20:02 <karsten> heh
15:20:12 <dennis_jackson> But the other distributions do not
15:20:34 <karsten> I ran out of ideas why this is the case.
15:20:50 <dennis_jackson> I then wanted to work out what was going on with the different parameters
15:20:55 <karsten> my current best guess is that there's something strange going on with the network setup.
15:21:06 <karsten> ah, go on. didn't want to interrupt.
15:21:08 <dennis_jackson> so I started looking at start2req, req2fb and the sum of the build times.
15:21:43 <dennis_jackson> There's a strange lack of correlation, at least to my mind
15:21:48 <karsten> yeah...
15:22:04 <dennis_jackson> I would naively expect start2req to be about the same as the sum of the build times
15:22:19 <karsten> circuits might have been built before the request came in.
15:22:29 <dennis_jackson> and req2fb to be correlated with start2req
15:23:03 <karsten> do you know where to find the onionperf analysis files?
15:23:25 <karsten> https://op-hk2.onionperf.torproject.net:8443/htdocs/
15:23:33 <karsten> just in case you want to dig deeper.
15:23:43 <dennis_jackson> Thanks
15:24:09 <dennis_jackson> Anyway, the correlations are much higher for nl than for hk
15:24:14 <karsten> but yes, I was actually looking for correlations there, too, but didn't find much.
15:24:49 <dennis_jackson> ['op-nl2,public,start2req,total_build', 0.6297887235661617]
15:25:03 <dennis_jackson> ['op-hk2,public,start2req,total_build', 0.4421435991718628]
15:25:11 <dennis_jackson> ['op-us2,public,start2req,total_build', 0.43186586805012234]
15:25:30 <dennis_jackson> Final column is the spearman correlation coefficient
15:25:54 <dennis_jackson> So looks like nl is using the circuits quicker maybe? But unsure without looking at the detailed files
15:26:19 <karsten> there are also the raw logs.
15:26:22 <dennis_jackson> I didn't find much conclusive, just lots of strange things. I plan to look a bit at the onionperf doc to improve my mental model a bit.
15:26:32 <karsten> https://op-hk2.onionperf.torproject.net:8443/tgen-client/log_archive/
15:26:39 <karsten> https://op-hk2.onionperf.torproject.net:8443/tor-client/log_archive/
15:27:19 <karsten> okay, let me add to the pad that you're planning to look more.
15:27:31 <karsten> would it make sense to start a second set of measurements nearby?
15:27:57 <dennis_jackson> Can't hurt, provided its not too much hassle.
15:28:09 <karsten> I don't know for hong kong.
15:28:20 <karsten> but we should easily get another machine in florida.
15:28:39 <karsten> and the netherlands.
15:28:44 <karsten> I'll try that.
15:29:07 <karsten> okay!
15:29:10 <dennis_jackson> okay, cool :)
15:29:15 <karsten> thanks for looking into this!
15:29:27 <dennis_jackson> np, its a nice change of pace from contact tracing
15:29:32 <karsten> heh
15:29:40 <karsten> moving on to:
15:29:41 <karsten> Reduce the number of 50 KiB downloads (#34023)
15:30:00 <karsten> what are the general thoughts on doing just the 5 MiB downloads?
15:30:21 <karsten> every five minutes, that is.
15:30:55 <dennis_jackson> I am in favour
15:31:09 <acute> that's 288 measurements per day?
15:31:29 <karsten> yep.
15:32:49 <acute> this is a good thing to do, I wanted to compare the number of measurements with how many we're doing at the moment
15:33:02 <karsten> the number of measurements stays the same.
15:33:10 <karsten> they'll all going to be 5 MiB downloads, though.
15:33:44 <karsten> by the way, the reason why there's no resp2done with the time from first to last byte in #34257 is that we don't have enough data for that.
15:33:57 <karsten> it just doesn't make much sense to do that for 50 KiB.
15:34:30 <karsten> okay, so no major concerns "OMG what are you doing"?
15:35:11 <karsten> sounds like it.
15:35:17 <karsten> great!
15:35:25 <acute> no major concerns here!
15:35:29 <karsten> :)
15:35:33 <karsten> next one is related:
15:35:34 <karsten> Reduce timeout and stallout values (#34024)
15:35:47 <karsten> the change is minor though.
15:35:58 <karsten> the idea is to reduce the timeout to 270 seconds to avoid overlapping measurements.
15:36:05 <karsten> and not use the stallout feature at all.
15:36:23 <karsten> but again, the impact is much smaller.
15:36:44 <karsten> I'd say if this seems like a crazy idea, be sure to shout either here or in the next couple hours on the ticket.
15:36:47 <karsten> otherwise we'll do it.
15:36:52 <dennis_jackson> Your proposal looks good to me. I think it would improve data quality
15:37:11 <karsten> to a very small extent in this case, but yes.
15:37:11 <dennis_jackson> It really just means that we consider failure to be after 3.5 minutes right?
15:37:21 <karsten> 4.5, yes.
15:37:30 <acute> it think it's an improvement
15:38:03 <karsten> cool!
15:38:15 <karsten> added a note.
15:38:17 <karsten> next is:
15:38:18 <karsten> Allow users to select Onion Service version to measure (#33434)
15:38:39 <karsten> basically, the idea is to kill v2 onion service measurements.
15:38:50 <karsten> they're almost exactly the same as v3 onion service measurements.
15:39:00 <karsten> I'd like to hear from dgoulet or asn about this.
15:39:39 <gaba> asn and dgoulet are in other meeting now.
15:40:01 <asn> karsten: just replied
15:40:02 <karsten> but if we do this and kill v2 onion service measurements, we'll have 1/2 measurements to public services and 1/2 to (v3) onion servies.
15:40:04 <gaba> :)
15:40:08 <karsten> oh hey!
15:40:31 <karsten> and you agree, nice. :)
15:40:36 <karsten> thanks for commenting!
15:40:47 <asn> o/
15:41:22 <karsten> removed the next topic because of time.
15:41:24 <karsten> Remove existing Tor control log visualizations (#34214)
15:41:34 <karsten> err, let me remove that, too.
15:41:49 <karsten> I'll bring them up next week.
15:42:13 <karsten> the next two are candidates for being picked up by somebody looking to write code.
15:42:16 <karsten> Fix message logging and filtering (#29369)
15:42:34 <karsten> I looked into the bug there and suggested a possible fix.
15:42:53 <karsten> now all that remains to be done is to write the code. :)
15:43:08 <karsten> same goes for:
15:43:09 <karsten> Combine multiple analysis files into single data set (#34191)
15:43:28 <karsten> except that that one is an enhancement, not a defect.
15:43:56 <karsten> we don't have to assign them now.
15:44:29 <karsten> just putting this out here if someone's looking for a task that doesn't involve diving deep into the code first.
15:45:13 <karsten> okay, if there are no questions, moving on?
15:45:41 <karsten> doing so.
15:45:42 <karsten> Python style
15:45:59 <karsten> this came up in the latest review.
15:46:11 <karsten> thanks, acute, by the way for putting all those effort into that review!!
15:46:16 <karsten> that*
15:46:41 <acute> :D
15:46:57 <karsten> what are the tools I should be using in the future to write code that complies with generally accepted style guides?
15:47:10 <karsten> what are other teams using?
15:48:02 <phw> karsten: there's flake8, which i use for bridgedb
15:48:20 <jnewsome> I think that's what we use in shadow as well
15:48:25 * jnewsome is checking...
15:48:37 <acute> pylint, yapf, black and pyflakes were mentioned on the ticket
15:49:20 <jnewsome> yeah, we use flake8 for enforcement. https://github.com/shadow/shadow/blob/master/.github/workflows/lint_python.yml
15:49:26 <jnewsome> (we = shadow)
15:49:45 <karsten> how much do they disagree?
15:49:51 <karsten> like, what if we use all of them?
15:49:59 <acute> flake8 seems to be a wrapper on top of pyflakes and pycodestyle, can you use it to reformat code?
15:50:35 * jnewsome doesn't know
15:51:24 <karsten> how about we try flake8 for the moment?
15:51:31 <acute> yapf and black are both formatters, it looks like the other tools are checkers
15:52:02 <karsten> I think black is, yes.
15:52:34 <acute> same for yapf
15:53:10 <karsten> okay. I'll try them out and bring this topic back to the agenda next week.
15:53:24 <karsten> (feel free to do the same.)
15:53:46 <karsten> last topic for today:
15:53:47 <karsten> Reviews
15:54:04 <karsten> I don't think there are any open reviews this week.
15:54:16 <karsten> we covered a few tickets currently under review.
15:54:32 <karsten> the part about cc'ing metrics-team is a minor issue, though.
15:54:56 <karsten> the thing is, if we don't cc metrics-team, updates are not being sent to metrics-bugs@.
15:55:20 <karsten> it's a minor issue, because I'll see updates eventually, but it's the reason why it takes me longer to respond.
15:55:42 <karsten> I hope we'll soon switch to gitlab and this problem will go away. ;)
15:55:49 <dennis_jackson> +1
15:56:04 <karsten> anyway, now you know. if you remember, put metrics-team in the cc field.
15:56:17 <karsten> anything else for today?
15:56:25 <acute> not from me!
15:56:28 <dennis_jackson> nothing from me
15:56:32 <jnewsome> nope
15:56:43 <phw> shall we add metrics-team to trac's auto-cc list?
15:56:52 <phw> for onionperf etc., that is
15:56:52 <karsten> that's for new tickets?
15:57:02 <karsten> we should, yes.
15:57:08 <phw> yes, for new tickets
15:57:21 <karsten> good idea, I'll do that.
15:58:02 <karsten> okay! thanks for a stuffed but very productive meeting, everyone!
15:58:19 <karsten> and thanks for working on making onionperf better!!
15:58:26 <phw> o/
15:58:29 <jnewsome> o/
15:58:30 <acute> o/
15:58:30 <dennis_jackson> :) o/
15:58:31 <karsten> talk to you next week, same time, same place. bye! o/
15:58:38 <gaba> o/
15:58:42 <karsten> #endmeeting