15:01:59 <karsten> #startmeeting metrics team meeting
15:01:59 <MeetBot> Meeting started Thu Feb 13 15:01:59 2020 UTC.  The chair is karsten. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:01:59 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
15:02:04 <gaba> :)
15:02:05 <gaba> thanks
15:02:28 <karsten> I added a few more topics to gaba's topics.
15:02:30 <karsten> do we have more?
15:03:34 <acute> hi everyone!
15:03:39 <karsten> hi acute!
15:04:06 <karsten> if not, shall we start?
15:04:17 <gaba> sure
15:04:23 <karsten> Review tasks from roadmap session (ticket creation, old cards in trello)
15:04:34 <karsten> what remains to be done?
15:04:37 <irl> i see this topic is so important we will do it twice
15:04:49 <karsten> just to be sure, yes.
15:04:54 <gaba> :)
15:05:03 <irl> i have created the exit scanner tickets, but did not set the keyword, i can do that after meeting
15:05:11 <karsten> I created tickets with the keyword.
15:05:12 <dennis_jackson> I have added a topic for Metrics w.r.t network health / metrics of our metrics
15:05:19 <gaba> i can st the keyword for them as I did not set the keyword for the others
15:05:21 <karsten> sounds good, dennis_jackson.
15:05:25 <gaba> I can do it for all of them
15:05:29 <gaba> irl ^
15:05:47 <irl> gaba: ok, but only the ones on the roadmap
15:05:54 <irl> not the extra ambitious ones we are not doing
15:05:58 <gaba> right
15:06:14 <gaba> I will do the tasks in that list today
15:06:36 <gaba> irl: you created all tickets that says NeedsTicket in https://pad.riseup.net/p/0K2Q5uuN0CbjbOGxJQsL-keep
15:06:37 <irl> ok, i will review tomorrow and let you know if i think anything is still out of sync
15:06:39 <gaba> right ?
15:07:08 <gaba> sorry, that is karsten
15:07:25 <karsten> I created tickets for O3.1 and O3.2.
15:07:33 <irl> acute has been creating tickets for onionperf work
15:07:43 <gaba> ok
15:07:47 <irl> and updating the pad with ticket numbers
15:08:09 <acute> gaba: I have been creating tickets for O1/O2
15:08:15 <irl> but again, i suggested not to add roadmap keywords because i wasn't sure what the most helpful keyword would be
15:08:33 <gaba> ok. I will go through them
15:08:37 <irl> ideally it is something that helps gaba to keep track
15:08:52 <karsten> acute: did you create tickets for all lines saying NeedsTicket?
15:09:00 <irl> would it be a good idea for acute to ping you once all of the tickets are made gaba?
15:09:13 <gaba> I will create tickets for the objectives and keep track of all this proposal tickets in the wiki page so far: https://trac.torproject.org/projects/tor/wiki/org/sponsors/Sponsor59
15:09:19 <gaba> yes irl, acute. thanks
15:09:35 <acute> karsten: no, not yet - it would be good to decide where we create the OP tickets (trac or gitlab)
15:09:51 <irl> i think we abandon gitlab
15:09:53 <karsten> okay, that's also a topic for today, but we can talk about it now.
15:10:04 <gaba> acute: for now we will do everything in trac until transition is oficial
15:10:09 <gaba> we can talk later about it
15:10:10 <gaba> yes
15:10:57 <karsten> everything happy with this?
15:11:01 <karsten> everyone*
15:11:07 <gaba> \o/ happy
15:11:07 <acute> yes!
15:11:10 <irl> (:
15:11:19 <karsten> great! :)
15:11:33 <karsten> anything else on this topic?
15:11:48 <karsten> oh, old cards
15:12:00 <irl> we should review it again next week and make sure we did make all the tickets and that gaba is happy with the keyword distribution
15:12:21 <gaba> yes
15:12:33 <karsten> should we clean up the trello thing and remove everything we're _not_ going to do in Q1?
15:12:48 <irl> i didn't touch the trello yet apart from existing onionperf tickets
15:12:52 <gaba> karsten: yes. I will do it and we can review next week
15:12:59 <irl> ok
15:13:02 <karsten> okay.
15:13:12 <karsten> next topic?
15:13:18 <gaba> yes
15:13:23 <karsten> Check on tickets from network health
15:13:46 <karsten> those tickets look like placeholders to me.
15:13:56 <gaba> I want to see if there is any work that metrics needs to do there
15:14:09 <gaba> this is something geko will be working on
15:14:14 <irl> the problem is that network health is still a poorly defined team
15:14:30 <irl> and we need to have a clearer idea of the demarcation line between metrics and network health
15:14:33 <gaba> irl: what do you mean?
15:14:37 <karsten> it's unclear from these tickets whether there's going to be something for the metrics team to do.
15:14:44 <gaba> ok
15:14:49 <dennis_jackson> I am curious about the overlap / split as well.
15:14:59 <karsten> but should we still add more work to Q1 at this point?
15:15:11 <gaba> karsten: I do not think so
15:15:30 <karsten> let's watch these tickets and see what work they produce for Q2 or later.
15:15:40 <gaba> ok
15:16:21 <gaba> related to network health team work I think geko will coordinate but tehre is work from other teams there. like dgoulet from network team and ggus are involved in it.
15:17:32 <karsten> alright. moving on?
15:17:50 <gaba> yes
15:17:58 <karsten> Proposal 313 - https://lists.torproject.org/pipermail/tor-dev/2020-February/014158.html
15:18:26 <karsten> I replied with some thoughts.
15:18:51 <karsten> are there open questions to us?
15:19:07 <karsten> or what's the question?
15:19:29 <gaba> mostly to see if anything related to that needs to be discuussed here or we are fine
15:19:37 <karsten> I think we're fine.
15:19:41 <gaba> ok
15:20:13 <karsten> Metric queries for GoodBadISPs https://dip.torproject.org/torproject/web/community/issues/72#note_483
15:20:38 <karsten> looks like Relay Search questions.
15:20:46 <irl> i was in this meeting
15:20:54 <karsten> ah, indeed, it says irl.
15:21:06 <irl> no one has sent an email yet, so i assume they are doing other stuff
15:21:16 <irl> last update there was before the meeting, so i guess no update
15:21:42 <gaba> ok
15:22:13 <karsten> okay. moving on then?
15:22:13 <irl> that PR doesn't implement any of my suggestions
15:22:19 <karsten> ah, sorry.
15:22:36 <irl> we need better data, not just blocks of english text
15:22:55 <irl> otherwise we have to build natural language processing and sentiment analysis systems to know if an isp is good or bad
15:23:18 <irl> but ok, this is not on our roadmap and maybe they are working on this
15:23:32 <gaba> yep
15:23:36 <karsten> sounds good.
15:23:39 <irl> i'm ok moving on
15:23:54 <karsten> moving on.
15:23:55 <karsten> Monthly reports, are they worth the effort?
15:24:16 <dennis_jackson> Uh, there are monthly reports? On metrics?
15:24:27 <karsten> there have been, until mid-2019.
15:24:43 <karsten> they took a few hours to write.
15:24:48 <irl> i struggle to keep up with tor-project@
15:24:52 <dennis_jackson> Ah okay, I was worried I was missing a mailing list. That's exactly when I became engaged with Tor
15:25:01 <irl> i don't think i've read a full week's worth of meeting reports in over a year
15:25:28 <karsten> same here, I'm not reading very many tor-project@ reports.
15:25:44 <irl> i guess we are the wrong people to know if they are useful or not
15:25:45 <karsten> so, my suggestion was:
15:25:47 <gaba> but I think other people are reading reports
15:25:59 <gaba> and they are available online as a way to understand what metrics is doing
15:26:06 <karsten> gaba and I were talking about changing the roadmap cycle from 3 months to 1 month.
15:26:31 <karsten> where we would not need weeks for creating a roadmap, obviously.
15:27:03 <karsten> that would be for the time after Q1 has ended. not affecting the current roadmap.
15:27:25 <irl> i think with only two people both over-capacity, 3 months is a long time to plan for
15:27:44 <karsten> now, when we have a roadmap for 1 month, it might be easier to write a quick summary for that month.
15:27:55 <irl> yeah that makes sense
15:28:13 <dennis_jackson> I just looked at an old report from July 2016. To me, it seems really nice to have. Even if its just tickets opened / closed / worked on (and what's coming next)  It helps people find where the coal face is for the team.
15:28:51 <karsten> good to hear that they have been useful.
15:28:54 <gaba> yes
15:29:15 <gaba> and we can add stuff to the report the last meeting of the month
15:29:23 <gaba> not having to do it only one person
15:29:30 <irl> i used to do these but apparently haven't done them for a long time https://iain.learmonth.me/blog/2018/2018w33/
15:29:54 <gaba> I can take the lead on setting up the space for the report each month
15:30:15 <irl> for thiings like the feedback process being able to remember what i did and when i did it, such reports would be useful too
15:30:52 <gaba> ok
15:31:07 <irl> interesting that my last weekly report was just before i did a week of travel, i guess i never caught up after i got back
15:31:10 <karsten> how about we write a 3-month summary for Q1 and then monthly reports starting in april?
15:31:16 <irl> ok
15:31:20 <gaba> ok
15:31:22 <gaba> sounds good
15:32:12 <karsten> alright. moving on?
15:32:20 <irl> ok
15:32:34 <karsten> GitLab migration and whether OnionPerf should stay or not
15:32:42 <karsten> already discussed, I think.
15:32:48 <irl> gitlab doesn't even work
15:32:57 <irl> you can't make merge requests
15:33:05 <irl> so i think there are no arguments for gitlab at this stage
15:33:19 <karsten> okay.
15:33:41 <karsten> I'm very much looking forward to a trac replacement, but I'm happy to wait.
15:34:04 <irl> right
15:34:08 <gaba> hiro is working on new server/configuration today
15:34:17 <karsten> thanks, hiro. :)
15:34:18 <gaba> and yes, I agree that is better to wait
15:34:29 <gaba> until we have something stable and migrated
15:34:32 <irl> do we need to rescue the issues back to trac quickly?
15:34:54 <irl> gitlab still has the onionperf issues
15:35:07 <irl> we are in progress of moving them back to trac
15:35:10 <gaba> irl: yes, I think we should use trac still
15:35:16 <gaba> otherwise it will be hard to migrate
15:35:19 <irl> i mean, is the gitlab data going to disappear?
15:35:25 <acute> ok, so remaining tickets that need to be created for OP should go in trac
15:35:26 <irl> like when hiro works on it today
15:35:30 <gaba> if we have data there we will need to move it...
15:35:41 <irl> do we have hours before we lose everything?
15:35:45 <irl> is it already gone?
15:35:51 <gaba> ha, no
15:36:19 <irl> acute: we might just reopen the closed issues for the ones that we migrated
15:36:26 <gaba> it will not be gon w without talking with people that may be using it
15:36:32 <irl> ok cool
15:36:48 <karsten> can you put the Trac component back, or do you need help with that?
15:36:51 <gaba> sorry about this. I should have more clear from the beginning not to use gitlab for trac replacement yet
15:37:06 <irl> i think i can do it
15:37:12 <irl> everyone is a trac admin these days
15:37:20 <karsten> makes sense, right?
15:37:31 <karsten> okay.
15:37:36 <karsten> next topic?
15:37:44 <irl> (component is back)
15:37:45 <irl> yes
15:37:54 <acute> thanks!
15:37:58 <karsten> Alternative to MaxMind's GeoLite2 database (#32978)
15:38:10 <karsten> I think I found something that we can use.
15:39:02 <karsten> I did a quick comparison of our current data to the new data source, and that was promising.
15:39:23 <karsten> I got feedback from a few relay operators, mostly positive.
15:39:39 <karsten> now, I'm not sure about legal questions and licenses and annoyances like that.
15:40:04 <karsten> I'll bring this up at the next vegas meeting to ask for legal stuff.
15:40:27 <gaba> ok
15:40:30 <irl> it seems to be similar enough, i guess we will get to know where it has its pros and cons
15:40:35 <karsten> if that doesn't bring up major concerns, I'll move this forward on the ticket.
15:41:03 <irl> do you have the database now?
15:41:09 <karsten> yes.
15:41:22 <irl> what is 44.190.21.1?
15:41:49 <karsten> 750654721, right?
15:42:05 <irl> huh?
15:42:11 <karsten> integer
15:42:23 <karsten> just hoping I didn't screw up the conversion.
15:42:38 <irl> i have never configured an ip address by integer
15:43:11 <karsten> US.
15:43:17 <irl> err no
15:43:29 <irl> wrong continent
15:43:31 <karsten> heh
15:43:44 <irl> but ok this is a strange one
15:44:15 <karsten> I can share details with you. I just don't want to announce this publicly until we have a rough idea whether we can use it at all.
15:44:28 <irl> ok, by email is fine and i will see if i can take a look
15:44:34 <karsten> great!
15:44:51 <karsten> hopefully I can move this all back to the ticket by the weekend.
15:45:06 <karsten> or early next week.
15:45:09 <karsten> okay, moving on?
15:45:16 <irl> ok cool
15:45:29 <karsten> Onionoo 8.0 release and deployment preparation
15:45:38 <karsten> we said we'd deploy 8.0 in 7 days.
15:45:55 <karsten> can you review the changes before that?
15:46:02 <karsten> they're small, IIRC.
15:46:11 <irl> i am working through reviews
15:46:16 <irl> i would say probably tes
15:46:18 <irl> yes
15:46:48 <karsten> I'm not sure if everything's under review yet. if not, I'll prepare it for early next week.
15:46:56 <karsten> cool!
15:47:15 <karsten> next and last topic?
15:47:40 <karsten> Metrics of Metrics / Network Health
15:47:44 <karsten> dennis_jackson: ^
15:48:03 <dennis_jackson> This is in part brought on by the discussion in #33076
15:48:11 <dennis_jackson> And also the earlier Network Health team tickets
15:48:58 <dennis_jackson> I realise the roadmap is already pretty intense for the next 3 months - so this is more about ideas and future direction than anything else
15:49:43 <dennis_jackson> So - what kind of metrics do we check about the metrics we gather?
15:50:16 <irl> do you mean like to verify the metrics, or instrumentation of the software we run to generate them?
15:50:57 <dennis_jackson> Both really. For example, the uptime of the four onionperf servers is in the latter category
15:51:09 <dennis_jackson> And the number of successful measurements in the last 24 hours in the former category
15:51:34 <dennis_jackson> Are there targets (of any kind) for these things? Documents with the goals set out?
15:51:48 <irl> both of these are metadata you can derive from the onionperf data
15:51:58 <irl> i am a big fan of in-band metadata
15:52:17 <irl> we don't have anything that looks like an SLA or targets, other than doing the best we can
15:52:25 <dennis_jackson> Yes, absolutely, my point is more about a) is anything / anyone looking at this? b) Does Tor Metrics have any kind of targets for them?
15:52:35 <dennis_jackson> Ah okay
15:52:38 <karsten> we have several checks in collector to warn us if anything looks stale.
15:52:44 <irl> we are working on some operations documentation that starts to document what to do when things go wrong, and what we check
15:52:52 <dennis_jackson> Would something like this be helpful?
15:52:53 <karsten> but, best effort, not systematically.
15:53:01 <acute> sounds like a fun grafana dashboard
15:53:03 <dennis_jackson> I worry the Network Health team want to know what Metrics can offer
15:53:12 <dennis_jackson> And Metrics want to know what Network Health need
15:53:45 <dennis_jackson> Refining some kind of target / spec might be the way to help out both teams.
15:54:10 <dennis_jackson> a way*
15:54:52 <dennis_jackson> Is something that might be useful to move towards? In 3,6 or 12 months time?
15:55:20 <dennis_jackson> I can also see arguments that it might be a distraction, not turn into a useful output, be quickly out of date, etc.
15:55:29 <irl> i don't think this would be useful to metrics
15:55:40 <irl> but that doesn't mean we shouldn't do this, if it would be useful to someone else
15:55:46 <irl> but they have to make a case for that
15:56:13 <karsten> yep.
15:57:04 <karsten> does that make sense?
15:57:16 <karsten> we're approaching the 60 minutes.
15:57:20 <gaba> I think it would be useful to understand how useful the data is. It is also something that may be helpful in the new metrics portal
15:57:28 <dennis_jackson> Okay, I think so.
15:57:46 <dennis_jackson> I am still getting a feel for how Tor works / how sub units relate / manage their time.
15:58:11 <dennis_jackson> And I appreciate how hugely busy the metrics team is with a lot of asks from all over the org
15:58:39 <karsten> :)
15:58:59 <karsten> alright! let's end this meeting in time!
15:59:02 <karsten> #endmeeting