14:00:03 <karsten> #startmeeting metrics team
14:00:03 <MeetBot> Meeting started Thu Sep  1 14:00:03 2016 UTC.  The chair is karsten. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:00:03 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
14:00:07 <iwakeh> Hi karsten!
14:00:11 <Sebastian> hi
14:00:12 <karsten> hello, who's here for the metrics team meeting?
14:00:15 <karsten> hi iwakeh and Sebastian!
14:00:22 <iwakeh> Hi Sebastian!
14:00:35 <karsten> https://pad.riseup.net/p/3M7VyrTVgjlF <- agenda pad
14:00:44 <karsten> it's a shiny new pad.
14:01:31 <iwakeh> Why new?
14:02:41 <karsten> ermm, because the old one still had a few chat lines that I didn't want to be reminded of over and over.
14:02:49 <iwakeh> ok ;-)
14:02:50 <karsten> also, because new pads are better? :)
14:03:03 <iwakeh> the old should be erased I think.
14:03:24 <iwakeh> any objections? pads are ephemeral
14:03:29 <karsten> how do you erase a pad?
14:03:38 <iwakeh> empty and drop?
14:04:02 <karsten> not sure how to drop.
14:05:21 <iwakeh> riseup destroys after 30 days of inactivity.
14:05:41 <karsten> ok.
14:05:52 <karsten> sounds like it'll solve itself.
14:06:01 <iwakeh> its empty now.
14:06:04 <karsten> thanks.
14:06:12 <karsten> so, what's for today?
14:08:02 <karsten> (want to add something to the agenda?)
14:08:14 <iwakeh> no, all there i think.
14:08:17 <karsten> ok!
14:08:22 <karsten> * Berlin meeting Sep 9--11 (karsten)
14:08:44 <karsten> we'll meet in person in ~7 days from now!
14:08:57 <iwakeh> yeah :-)
14:09:00 <karsten> what can we do to make that time as usefully spent as possible?
14:10:06 <iwakeh> well, we have a loose plan for saturday.
14:10:19 <Sebastian> (I'll join the meeting btw)
14:10:33 <iwakeh> fine!
14:10:34 <karsten> Sebastian: the berlin meeting? cool!
14:10:39 <Sebastian> yep
14:12:06 <iwakeh> Sat, Sep 10th 10:00 - 18:00: metrics day
14:12:09 <karsten> so, we'll want to prepare that open session in the morning a bit.
14:12:16 <iwakeh> right.
14:12:37 <karsten> which can happen in part the day before.
14:12:50 <iwakeh> oh?
14:12:57 <karsten> though we can easily start with the prepartion this week.
14:13:34 <karsten> what I mean: we can start making plans in the next couple of days and then talk about the open session next friday in person.
14:13:38 <karsten> to finalize it.
14:14:13 <iwakeh> sounds ok.
14:14:40 <karsten> other than that, we might want to discuss the exonerator database while we're there and also webstats.
14:14:50 <karsten> after all, we'll have GeKo there, too.
14:15:49 <karsten> so, I was hoping to think more about the berlin meetup the last days, but then the Tonga/Bifroest transition kept me busier than I had thought.
14:16:04 <iwakeh> What topics should be in the open session?
14:16:05 <karsten> how about I send some ideas via email in the next few days, and then we go from there?
14:16:14 <iwakeh> ok.
14:16:25 <karsten> well, I think we should tell people what our plans are for the next 6~9 months.
14:16:28 <karsten> 6-9
14:16:44 <karsten> and what's out of scope for that time period.
14:17:04 <karsten> and then we should ask what they'd like us to do, either as part of MOSS work or beyond that.
14:17:15 <karsten> and maybe even get suggestions for the time after MOSS.
14:17:27 <iwakeh> sounds like a plan.
14:17:58 <karsten> cool. feel free to send more suggestions my way if something comes up.
14:18:17 <karsten> okay, next topic?
14:18:25 <iwakeh> yep
14:18:30 <karsten> * August monthly report, September plans (karsten)
14:19:12 <karsten> there, I pasted something on the pad.
14:19:45 <karsten> including pointers for myself to write a bit more about these items.
14:19:57 <karsten> want to throw in your items?
14:20:06 <karsten> or email them to me?
14:20:32 <iwakeh> rather email.
14:20:37 <karsten> yep, sounds good.
14:20:55 <karsten> ah, september plans is also in there.
14:21:08 <iwakeh> CollecTor 110
14:21:15 <karsten> remind me, what did we put down for september, after working on collector in july and august?
14:21:42 <karsten> right, collector 1.1.0.
14:22:24 <iwakeh> we planned to start
14:22:36 <iwakeh> quite a few things already in Aug.
14:22:39 <iwakeh> like
14:23:15 <iwakeh> privacy and webdesign and
14:23:43 <karsten> hrmm
14:23:46 <iwakeh> onion perf, but no particular ms for sept.
14:23:57 <iwakeh> actually, we still need
14:24:18 <iwakeh> to re-plan the later addition of measurable mstones.
14:24:35 <iwakeh> see https://trac.torproject.org/projects/tor/wiki/org/sponsors/SponsorX#Tasks
14:25:02 <karsten> oh wait,
14:25:13 <karsten> tor can only invoice complete tasks, right?
14:25:21 <iwakeh> right.
14:25:30 <iwakeh> so collector?
14:25:43 <karsten> yes, let's finish that.
14:25:48 <karsten> so that tor can invoice for it.
14:26:03 <iwakeh> number 2 could also be finished?
14:26:30 <karsten> quite easily, yes.
14:27:12 <iwakeh> actually, I'm not 100% sure, if partial invoicing is fine?
14:27:13 <karsten> you mean finish 2 before 1?
14:27:24 <iwakeh> no, just in september.
14:27:26 <karsten> I'm not certain. it's in the mails somewhere.
14:27:57 <karsten> well, I wonder how hard the CollecTor sync feature will be, and when we'll be comfortable enough to enable it.
14:28:10 <karsten> though deploying 1.0.1 was a big step forward there.
14:28:26 <iwakeh> enable doesn't mean 100% uptime, does it?
14:28:52 <karsten> well, it would be bad if it breaks just because we're setting it up too early.
14:28:58 <karsten> but I'm not sure.
14:29:11 <iwakeh> and having my mirror running synch won't hurt the main collector.
14:29:14 <karsten> so, we could finish 1 or 2 soon.
14:29:26 <karsten> ah, right.
14:29:31 <karsten> doesn't have to be bidirectional.
14:29:36 <iwakeh> right.
14:29:39 <karsten> good point.
14:30:02 <karsten> alright, sounds like a plan.
14:30:27 <iwakeh> I could mail a newer planning suggestion?
14:30:40 <karsten> yes, please!
14:31:03 <karsten> we should also keep in mind that half of my september is traveling, travel preparation, meeting, etc.
14:31:14 <karsten> (not in that order)
14:31:15 <iwakeh> true, and october too.
14:31:28 <karsten> what's in october?
14:31:35 <iwakeh> the dev-meeting?
14:31:44 <karsten> ah, that's last week of september.
14:31:59 <iwakeh> oh, well even better.
14:32:35 <karsten> alright, let's plan more via email.
14:32:40 <karsten> next?
14:32:44 <iwakeh> yep.
14:32:48 <karsten> * Help with Tonga/Bifroest transition aftermath (karsten)
14:32:51 <karsten> *sigh*
14:32:59 <iwakeh> help needed?
14:33:08 <karsten> #20049
14:33:14 <karsten> a bit, yes.
14:33:25 <karsten> so, onionoo should survive okay.
14:33:35 <karsten> it's a bit confused by bridge network statuses coming from two sources.
14:33:36 <iwakeh> review?
14:33:59 <karsten> as in, it doesn't consider all bridges as running, just the ones from the status that happened to be published last.
14:34:09 <karsten> no review for onionoo, because there's no new code.
14:34:31 <karsten> I figured it's okay to have slightly onionoo clients for a couple of days.
14:34:45 <karsten> this will heal by itself on saturday when Tonga goes away.
14:34:46 <iwakeh> ?
14:34:57 <karsten> the running part.
14:35:02 <iwakeh> ' it's okay to have slightly onionoo clients '
14:35:12 <Sebastian> slightly broken? ;)
14:35:13 <karsten> ah, slightly *confused* onionoo clients.
14:35:18 <iwakeh> ok :-)
14:35:28 <karsten> so, this will affect ~50% of bridges.
14:35:39 <karsten> which will show up as non-running even though they're running.
14:35:49 <karsten> I figured that's acceptable.
14:35:54 <karsten> temporary problem.
14:36:06 <karsten> now, the metrics website is more difficult.
14:36:15 <karsten> we're using bridge data in two modules: legacy and clients.
14:36:26 <karsten> I think I have a fix for the legacy module in #20049.
14:36:48 <karsten> that code might be a bit rough yes, because I didn't even have the chance to read it myself after writing it.
14:37:06 <karsten> but we'll have to deploy that really soon (tomorrow?), because metrics-web isn't updating right now.
14:37:25 <karsten> having a review would be cool!
14:37:28 <iwakeh> ok.
14:37:35 <karsten> the other module is clients.
14:37:45 <karsten> I haven't looked yet, but it might be that it'll work just fine.
14:37:50 <karsten> without new code.
14:38:08 <karsten> I hope to be able to look this evening.
14:38:23 <karsten> after the other 90-minute meeting I have in ~60 minutes.
14:39:00 <karsten> however, if this requires new code, it would be cool to have a quick review of that.
14:39:12 <iwakeh> I reserve review time for tomorrow morning, too.
14:39:17 <karsten> thanks!
14:39:25 <karsten> okay, next?
14:39:32 <iwakeh> yes
14:39:36 <karsten> * Collaboration with community team on relay operator lists (karsten)
14:39:49 <karsten> maybe you saw the requests on tor-project@ and tor-relays@?
14:39:55 <iwakeh> yes I did.
14:40:12 <karsten> it seems that dgoulet is going to do the coding here. yay!
14:40:28 <iwakeh> python it was?
14:40:29 <karsten> I was wondering if you'd want to guide him to code the right thing?
14:40:36 <karsten> yes, python is the plan, I think.
14:40:49 <karsten> but this is mostly about using the right data from onionoo.
14:40:59 <dgoulet> o/
14:41:04 <karsten> hello dgoulet! :)
14:41:21 <karsten> we also have the t-shirt script that's written in python and that uses onionoo data.
14:41:36 <dgoulet> yes
14:41:45 <karsten> but really, this is about brainstorming what can be done with onionoo data and on the other hand what people would need.
14:41:55 <karsten> maybe in the other order. ;)
14:42:02 <dgoulet> right but that would be at the next community meeting or now?
14:42:07 <karsten> not now.
14:42:22 <karsten> trying to talk iwakeh into helping with that, because I'm drowning in commitments..
14:42:25 <karsten> ;)
14:42:29 <dgoulet> oki
14:42:32 <iwakeh> ok :-)
14:42:46 <iwakeh> but, I'll need to read up on the requirements.
14:42:51 <iwakeh> a bit.
14:43:15 <karsten> cool! feel free to ask if anything's unclear in existing threads,
14:43:20 * Samdney from the community team is watching ;)
14:43:46 <karsten> but I think that requirements might not be written down anywhere yet.
14:43:49 <karsten> hi Samdney!
14:43:59 <Samdney> hi karsten!
14:44:36 <karsten> and it might even be that the next community team meeting next wednesday won't produce complete requirements, but it could be a good start.
14:44:58 <iwakeh> I just look at the threads in the ml and ask before that meeting.
14:45:05 <karsten> sounds great!
14:45:24 <karsten> would you want to post to the list that you'll be there for the meeting?
14:45:28 <karsten> or should I respond with something?
14:46:10 <iwakeh> that was the community list? I can do that.
14:46:26 <karsten> I mean on tor-project@.
14:46:37 <karsten> I think the community list is private.
14:47:04 <karsten> Re: [tor-project] Metrics to help relay operators?
14:47:05 <karsten> that one.
14:47:26 <iwakeh> ok. that's fine too. I just thought I read some
14:47:37 <iwakeh> note about switching to that list.
14:47:55 <Samdney> Btw, just saw flexlibris has opened a pad https://pad.riseup.net/p/metricsneeds
14:48:07 <karsten> yep. we did switch, because non-members cannot post to metrics-team@.
14:48:20 <karsten> team lists are a bit of a mess right now...
14:48:25 <karsten> nice, Samdney!
14:48:43 <karsten> okay, should we move on to the next topic?
14:49:04 <iwakeh> fine
14:49:05 <karsten> (we can't discuss requirements today..)
14:49:15 <karsten> * exonerator update (Sebastian)
14:49:24 <karsten> how's it going, mr. 48G RAM?
14:49:50 <Sebastian> so I played with the db a bit
14:50:20 <Sebastian> Basically, I think we can lift the requirement to search for specific dates only with a new db layout
14:50:44 <karsten> you mean search for date ranges?
14:50:50 <Sebastian> yes
14:50:55 <Sebastian> looking for a relay in a given /24 ipv4 network takes 12 milliseconds across the entire stored history
14:51:04 <Sebastian> compared to 27 seconds for the old database layout
14:51:07 <karsten> hehe
14:51:21 <Sebastian> this is all on my machine which is a bit more powerful than the server we'll host this on
14:52:06 <karsten> so, this is also a usability question.
14:52:18 <karsten> right now, there's a date field, and we return results +/- 1 day.
14:52:58 <karsten> how would this change? provide a from and a to field and return results in [from-1, to+1]?
14:53:35 <Sebastian> I think we can keep the current interface but if you don't enter a date you get date ranges back where there was a relay /it was an exit
14:54:02 <karsten> could be a lot of data in the technical details section.
14:54:03 <Sebastian> I've often scripted exonerator to look for months at a time individually because I only cared whether this IP address was ever a relay
14:54:12 <Sebastian> this is not nice on the db load ;)
14:54:48 <karsten> okay, but we can separate that change from the db redesign?
14:54:55 <Sebastian> of course
14:54:58 <Sebastian> it's just possible now
14:54:59 <karsten> because the primary reason to do that was to make it smaller and faster.
14:55:04 <karsten> okay, that's very cool.
14:55:12 <Sebastian> my primary motivation for helping is making it useful for me :)
14:55:16 <karsten> hah
14:55:25 <iwakeh> best reason :-)
14:55:30 <karsten> so, you have completed the database migration?
14:55:45 <Sebastian> No, I haven't done anything with exitlist stuff yet
14:55:50 <Sebastian> and I haven't written these functions
14:56:12 <karsten> ah, exitlist data is tiny compared to statuses.
14:56:12 <Sebastian> I just got done importing everything and running some toy queries shortly after the meeting started.
14:56:15 <Sebastian> and wanted to report.
14:56:32 <karsten> sounds great! :)
14:56:53 <karsten> let me know if there's something for me to be done?
14:57:13 <Sebastian> I hope it'll be done in Berlin
14:57:22 <Sebastian> nothing else from m
14:57:22 <Sebastian> e
14:57:34 <karsten> great! pity that we didn't include this in the MOSS project. ;)
14:57:47 <karsten> okay, I think we ran out of items.
14:57:50 <karsten> and time.
14:57:58 <karsten> anything else?
14:58:01 <velope> as far as i see, https://exonerator.torproject.org currently isn't linked from any metrics page (or anywhere even)
14:58:19 <karsten> velope: good point. there's even a ticket for that.
14:58:22 <iwakeh> https://trac.torproject.org/projects/tor/wiki/doc/ExoneraTor/Improvements
14:58:27 <karsten> not sure where it would be linked.
14:58:42 <iwakeh> https://trac.torproject.org/projects/tor/wiki/org/teams/MetricsTeam#Products
14:58:45 <velope> so is exonerator currently considered supported/operational?
14:58:58 <karsten> oh, yes.
14:59:50 <karsten> #17771
14:59:56 <karsten> want to add ideas there?
15:00:35 <karsten> okay, we really ran out of time.
15:00:42 <iwakeh> yes.
15:00:45 <velope> i'm not really concerned with the specfics, i was just surprised
15:00:46 <karsten> I cannot attend next week's meeting, because I'll be traveling.
15:00:50 <karsten> velope: ok.
15:01:04 <karsten> maybe others will be traveling, too. and we'll be meeting in person the days after.
15:01:11 <karsten> I suggest we skip next week's IRC meeting.
15:01:24 <iwakeh> right, have a meeting friday afternoon?
15:01:29 <iwakeh> in person.
15:01:32 <karsten> yes, sure.
15:01:55 <karsten> alright. thanks for a great meeting, everyone!
15:01:59 <karsten> #endmeeting