15:01:02 <karsten> #startmeeting metrics team
15:01:02 <MeetBot> Meeting started Thu Apr 13 15:01:02 2017 UTC.  The chair is karsten. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:01:02 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
15:01:08 <karsten> hi!
15:01:29 <karsten> shorter meeting than usual today, because I screwed up coordinating this meeting with another one.
15:01:45 <karsten> https://pad.riseup.net/p/3M7VyrTVgjlF <- agenda
15:02:03 <karsten> I saw Samdney, hiro, and iwakeh.
15:02:09 <hiro> hi everyone
15:02:26 <karsten> hiro: want to start with the first topic?
15:02:30 <karsten> - OnionPerf deployment (hiro)
15:02:32 <hiro> sure
15:02:37 <iwakeh> Hi hiro and karsten  and all else!
15:02:38 <karsten> cool!
15:02:41 <karsten> hi iwakeh!
15:02:51 <hiro> so it looks like measurements are finally consistent and we do not have timeouts
15:02:57 <karsten> yay!
15:03:00 <iwakeh> neat!
15:03:03 <hiro> which is good
15:03:18 <hiro> so last changes from rob worked pretty good
15:03:30 <hiro> so what would be missing now is state of op-ab
15:03:56 <hiro> I was chatting with irl end of last week and he was busy w something else, so I will try to catch up again on this maybe tomorrow or next week
15:04:03 <hiro> also old tor-perf vm
15:04:22 <karsten> old torperf vm?
15:04:24 <hiro> talked with ln5 too and explained the setup, have to see how is that going on
15:04:29 <karsten> ah, that's siv.
15:04:48 <karsten> there's also a vm where we run a torperf instance called, ermm, torperf.
15:04:48 * Samdney Is back, some connecting problems today.
15:04:56 <karsten> wb!
15:04:59 <hiro> wb?
15:05:03 <karsten> welcome back.
15:05:05 <hiro> ahh neat
15:05:34 <hiro> so I think that's all on op-side
15:05:35 <karsten> so, still a few steps remaining to have all onionperf instances up and running,
15:05:45 <karsten> but we have 3 by now, and they're working as expected.
15:05:47 <karsten> right?
15:05:52 <hiro> yes
15:06:12 <karsten> ah, can you delete the 2017-04-10 files?
15:06:17 <hiro> yep!
15:06:28 <karsten> then I'll update collector tarballs.
15:06:32 <karsten> thanks!
15:06:35 <hiro> i think in the end we will have 5, but the last 2 will take a little bit longer
15:06:41 <karsten> that's okay.
15:06:45 <hiro> I'll delete right now
15:06:50 <karsten> great!
15:06:55 <karsten> okay, moving on?
15:07:25 <karsten> ah, just one related thing:
15:07:42 <karsten> given that we took longer to get onionperf running and remaining issues resolved,
15:08:01 <karsten> I'm inclined to take the shortcut we discussed earlier and declare onion service measurements the new user model for sponsor X.
15:08:13 <iwakeh> yes.
15:08:26 <karsten> rather than including the more realistic website measurement. which we can still do, but which shouldn't block calling this deliverable done.
15:08:50 <iwakeh> It is a new user-model.
15:08:56 <karsten> yes, and it's useful.
15:09:01 <iwakeh> true.
15:09:15 <karsten> okay, great! moving on.
15:09:20 <karsten> - In-memory stats and report (iwakeh)
15:09:53 <iwakeh> the simulation was useful,
15:10:21 <iwakeh> and we should be cautious adding noise too generously.
15:10:27 <karsten> yep.
15:11:19 <iwakeh> The amount of noise (or the standard deviation)
15:11:34 <iwakeh> thereof increases by sqrt(r)
15:12:01 <iwakeh> where r is the number of reports (i.e. extra-info descr) we sum up.
15:12:16 <karsten> so, hmm,
15:12:29 <iwakeh> that's bad for low count
15:12:40 <karsten> should we look into alternative estimation methods that don't include r reports?
15:12:44 <iwakeh> many or medium number of reports countries.
15:13:06 <karsten> maybe something like we do for onion service stats where we use a trimmed mean?
15:13:17 <iwakeh> in that way
15:13:22 <karsten> so, r/2 reports. not a huge change, but potentially better.
15:13:25 <karsten> twice as good!!
15:13:35 <iwakeh> yes, this will need more work outside sponsorX.
15:14:03 <karsten> perhaps.
15:14:13 <karsten> what if we finish onionperf work and metrics-lib this month?
15:14:20 <iwakeh> hmm, well rather averaging
15:14:21 <karsten> for onionperf, see above.
15:14:27 <karsten> metrics-lib is just a blog post away.
15:14:38 <karsten> we would have two months for these stats.
15:14:46 <karsten> it seems most relevant.
15:14:51 <iwakeh> You're offline until 21.?
15:14:56 <karsten> yep!
15:15:12 <karsten> well, back on the 21st.
15:15:20 <iwakeh> ok.
15:15:25 * Samdney is also interested in stats topics :)
15:15:34 <iwakeh> should I focus on reviews?
15:15:52 <karsten> Samdney: good point, we should share simulation results soon.
15:16:13 <karsten> iwakeh: during this week? that would make sense, I think.
15:16:17 <iwakeh> this is next topic (already)
15:16:21 <karsten> yep.
15:16:25 <hiro> Karsten what if it takes more time for irl and ln5 to integrate their machines? would that be still ok?
15:16:32 <karsten> hiro: ah, yes.
15:16:42 <karsten> hiro: we said we'd replace 3 torperf instances with 3 onionperf instances.
15:16:48 <iwakeh> we have several deployments.
15:16:49 <hiro> ah that's ok
15:17:00 <karsten> we should still do the other 2, but we can call the deliverable done before that.
15:17:08 <iwakeh> yep.
15:18:13 <karsten> how about we put early simulation results on metrics-team@ today/tomorrow?
15:18:36 <karsten> my fault to start that as private thread.
15:19:42 <karsten> iwakeh: maybe you can respond to my latest comments, and I start a new thread on metrics-team@?
15:19:49 <karsten> (comments via email)
15:20:15 <iwakeh> sorry, just skimmed through that mail :-)
15:22:26 <karsten> iwakeh: so, we have until end of june for sponsor x things.
15:22:35 <iwakeh> yeah
15:22:46 <karsten> I don't want to rush putting in noise only to find out later that we can't handle it.
15:22:59 <karsten> but we could try to handle it over the next weeks,
15:23:03 <iwakeh> Correct, that's why I suggest waiting
15:23:11 <karsten> and either we figure out that we can, in which case we can put it in,
15:23:20 <iwakeh> the patches can
15:23:27 <karsten> or we figure out that we can't or are not sure yet, in which case we should hold that back.
15:23:32 <iwakeh> already pave the way for easy introduction of
15:23:44 <iwakeh> this noise, but we should not rush this now.
15:23:49 <karsten> agreed.
15:24:04 <karsten> the easy way out is to say we're not certain yet, and I believe we should keep that.
15:24:09 <iwakeh> and, such a discussion and changes and simulations will
15:24:27 <iwakeh> take way too much time from the core sponsor tasks.
15:24:55 <karsten> but which remain? :)
15:25:05 <karsten> I mean, certainly, the patches need to be written anyway.
15:25:23 <karsten> anyway, I need to run in 4 minutes.
15:25:29 <iwakeh> right :-)
15:25:33 <karsten> no need to decide this today!
15:25:48 <karsten> just saying that more work on stats is more important than more work on onionperf or metrics-lib.
15:26:01 <iwakeh> true.
15:26:08 <Samdney> +1
15:26:15 <karsten> - Tickets to review (karsten)
15:26:19 <karsten> https://trac.torproject.org/projects/tor/query?status=needs_review&component=%5EMetrics%2FOnionoo&component=%5EMetrics%2Fmetrics-lib&component=%5EMetrics%2FCollecTor&component=%5EMetrics%2FMetrics+website&col=id&col=summary&col=status&col=type&col=priority&col=milestone&col=component&order=priority
15:26:41 <karsten> I didn't look which of these require review by me.
15:27:00 <iwakeh> seems all on m list :-)
15:27:10 <karsten> cool! :D
15:27:21 <karsten> okay, good thing I'm out for a week and won't produce more tickets to review.
15:27:31 <iwakeh> hehe
15:27:31 <karsten> (thanks!)
15:27:36 <karsten> - (No?) meeting next week, meeting time 1 hour earlier? (karsten)
15:27:46 <iwakeh> two questions?
15:27:48 <karsten> so, I won't be around next week.
15:28:00 <karsten> I'd say let's skip next week.
15:28:04 <karsten> for meetings.
15:28:07 <hiro> fine by me
15:28:07 <iwakeh> ok.
15:28:14 <karsten> the second question is about the meeting time.
15:28:25 <karsten> to avoid situations like today, it would be good to have future meetings 1 hour earlier.
15:28:31 <karsten> 30 minutes would work, too.
15:28:35 <hiro> ok
15:28:44 <iwakeh> 30min
15:29:00 <karsten> quick poll. 1h or 30min?
15:29:11 <hiro> 30min
15:29:16 <iwakeh> 30minn
15:29:21 <iwakeh> -n
15:29:37 <karsten> okay!
15:30:00 <karsten> I'll announce the next meeting for 14:30 UTC in 2 weeks from now.
15:30:01 <Samdney> ok
15:30:05 <karsten> great!
15:30:11 <karsten> thanks, everyone!
15:30:36 <karsten> if any other questions remain, I'm around until tomorrow evening via email.
15:30:42 <karsten> bye!
15:30:44 <iwakeh> bye, bye, and happy holidays.
15:30:45 <Samdney> bye!
15:30:45 <hiro> bye!
15:30:49 <karsten> :)
15:30:52 <hiro> happy holidays :)
15:30:52 <karsten> #endmeeting