14:29:49 <karsten> #startmeeting metrics team
14:29:49 <MeetBot> Meeting started Thu Dec  7 14:29:49 2017 UTC.  The chair is karsten. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:29:49 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
14:29:59 <karsten> who is here for the metrics team meeting?
14:30:07 <irl_work> 
14:30:11 <karsten> hi!
14:30:21 <irl_work> hi!
14:30:25 <karsten> a world map... :)
14:30:28 <hiro> me
14:30:32 <karsten> hi hiro!
14:30:38 <hiro> hello
14:30:45 <karsten> (https://atlas.torproject.org/#map)
14:30:50 <karsten> nice!
14:31:08 <irl_work> don't credit me for the world map, the actual map code was written by ana
14:31:16 <irl_work> i just intergrated it
14:31:26 <karsten> https://storm.torproject.org/shared/Ou-1QRctynWbF4yedi-MfDsjImFMFSIEP20fbVGCPRa <- agenda pad
14:31:34 <karsten> who is ana?
14:31:38 <irl_work> my partner
14:31:51 <karsten> cool! say thanks. :)
14:31:55 <irl_work> i will (:
14:32:25 <karsten> time to add a "What has changed" section to the page. so many new things. :)
14:32:39 <karsten> we're still missing iwakeh. hmm.
14:32:48 <irl_work> there will be a blog post maybe next week on what has changed, perhaps in tutorial style
14:33:09 <irl_work> not started it yet though
14:33:17 <karsten> did you talk to tommy about that?
14:33:21 <irl_work> not yet
14:33:21 <karsten> or steph?
14:33:30 <karsten> because they asked for a blog post.
14:33:34 <karsten> this could be it.
14:33:46 <irl_work> i know, i was a little ill this week which has put a large dent in productivity
14:33:53 <karsten> :(
14:33:57 <irl_work> but that is where i plan to start
14:34:37 <karsten> okay, please let me know if we can help.
14:34:43 <irl_work> will do (:
14:35:52 <karsten> hi iwakeh!
14:36:00 <iwakeh> hi karsten!
14:36:35 <karsten> okay, shall we start?
14:36:39 <iwakeh> meeting pad loads slowly here
14:36:49 <iwakeh> yes, let's start.
14:36:52 <karsten> (we used the time to talk about relay search and a blog post.)
14:37:13 <karsten> * Proposal 288: Privacy-Preserving Statistics with Privcount in Tor (Shamir version) (karsten)
14:37:26 <karsten> (moved topics around a bit, b/c slow pad.)
14:37:34 <karsten> so, there's a new proposal.
14:37:49 <karsten> somebody should take a look and see how we're affected.
14:38:14 <karsten> it's on tor-dev@.
14:38:43 <karsten> and let me add the next one, too.
14:38:47 <karsten> * OONI's vanilla tor data analysis (karsten)
14:39:07 <karsten> the OONI team has finished some data aggregations that they shared internally on tor-team@.
14:39:28 <karsten> somebody should take a look and see how we can make graphs out of that for the metrics website.
14:39:52 <irl_work> on privcount, i doubt that i will have time to really get up to speed with this until 2018. i'd like to read the specs and proposals and maybe some code on the trip to/from ccc.
14:40:17 <karsten> sounds good!
14:40:26 <karsten> iwakeh writes on the pad that they'll take a look.
14:40:55 <irl_work> i am somewhat familiar with the OONI results format having used OONI for university research, i wonder if we could add a parser for metrics-lib for their json format
14:40:56 <karsten> more eyes == better
14:41:13 <karsten> in this case I think they
14:41:21 <karsten> 're providing CSV files.
14:42:07 <karsten> we explicitly said that we're not going to parse and aggregate their data, because of the amount of dev time that this would take.
14:42:23 <karsten> see the servers-ipv6 stuff. that's relatively simple, in theory, but it adds up to a whole lot of code.
14:42:46 <irl_work> so do we just plot what is in the CSV?
14:42:47 <karsten> so, for this step it's really just about taking their CSV files and making some useful graphs out of them.
14:42:51 <karsten> exactly.
14:42:57 <karsten> we can extend that in 2018.
14:43:07 <karsten> but for now, it's just plotting.
14:43:19 <karsten> welcome back, iwakeh!
14:43:20 <iwakeh> back again, sorry.
14:43:48 <karsten> backlog in your inbox.
14:43:57 * iwakeh looking ...
14:44:00 <irl_work> i'm concious of how much time i have left this year, but if it's just producing the code to plot then maybe that's ok
14:44:34 <karsten> maybe it's even less: just looking at the data, writing some prototype graphs, and say what will be useful.
14:44:47 <irl_work> that would be ok
14:45:02 <karsten> the R/ggplot2 code and metrics-web stuff (metrics.json) is something I could write quite easily.
14:45:20 <karsten> did you see arturo's email?
14:45:29 <irl_work> i've seen a thread but not read it in detail yet
14:45:33 <karsten> "The vanilla tor data analysis" from Nov 30.
14:45:42 * irl_work doesn't have email at the moment
14:45:48 <karsten> oh. :)
14:45:59 <karsten> ah, just not right now.
14:46:08 <irl_work> i'm in the office
14:46:14 <karsten> should I fwd, just in case?
14:46:22 <irl_work> please do, and i'll add it to my queue
14:46:27 <karsten> great!
14:46:50 <karsten> cool. next topic:
14:46:54 <karsten> * Monthly report draft (karsten)
14:47:18 <karsten> I put the draft on the pad.
14:47:41 <irl_work> sorry for dropping a link right in the middle and messing up the numbering
14:47:42 <iwakeh> well, looks like a lot :-)
14:47:44 <karsten> please take a look within the next, say, 24 hours, so that I can send it friday afternoon.
14:47:52 <karsten> iwakeh: it's a whole lot!
14:47:57 <iwakeh> true
14:47:59 <karsten> irl_work: no worries at all :D
14:48:28 <karsten> what's worse is that you dropped british english in the middle, and now I'm in conflict.
14:48:35 <irl_work> heh
14:48:39 <karsten> can I correct a native speaker...
14:48:58 <irl_work> we have this all the time submitting to conferences/journals
14:49:08 <irl_work> flip a coin
14:49:20 <karsten> hehe
14:49:24 <iwakeh> it might converge in a few years
14:49:40 <iwakeh> with many other <country-ism>s
14:49:50 <hiro> karsten report looks good to me w/ reg to saltstack
14:49:51 <karsten> soon it will all be Bad English.
14:49:58 <karsten> hiro: great!
14:50:07 <iwakeh> all fine.
14:50:22 <karsten> alright, next topic:
14:50:24 <karsten> * Review of #24260: Add metrics timeline events underneath graphs (karsten)
14:50:50 <irl_work> example: https://metrics.torproject.org/userstats-relay-country.html
14:51:03 <karsten> not sure how much this review matters, but it would be good to have a quick "ok" before putting it into master.
14:51:31 <iwakeh> it's neat!
14:51:37 <karsten> yes, it's already deployed.
14:51:41 <irl_work> i like it
14:51:45 <irl_work> i especially like the filtering
14:51:50 <irl_work> not looked at the code though
14:52:12 <iwakeh> filtering?
14:52:19 <karsten> the filtering is a start. it could certainly be better, but it might be something that evolves over time.
14:52:25 <karsten> filtering of news events shown.
14:52:38 <irl_work> userstats-bridge-transport.html doesn't filter
14:52:43 <irl_work> i mean, by transport
14:52:58 <karsten> it filters by country.
14:53:04 <irl_work> but yes, i can see this being useful and answering questions that i've seen often asked
14:53:07 <karsten> ah.
14:53:16 <karsten> yes, filtering by transport was tricky.
14:53:21 <iwakeh> ah, what's in the lists, not by column or tag, I see.
14:53:38 <karsten> no column sorting/filtering, right.
14:53:49 <iwakeh> not necessary.
14:53:56 <karsten> on the list for later. :)
14:54:01 <iwakeh> the way it works is a great improvement.
14:54:10 <iwakeh> yep.
14:54:16 <karsten> so, the review,
14:54:23 <karsten> can somebody take a quick look at the code?
14:54:34 <iwakeh> sure.
14:54:39 <karsten> cool!
14:54:59 <karsten> next one is related again:
14:55:00 <karsten> * Review of #24218 (database schema): Implement new metrics-web module for IPv6 relay statistics (karsten)
14:55:09 <karsten> it's just about the database schema.
14:55:18 <karsten> because as soon as I know it's final, I can import data.
14:55:36 <iwakeh> yes, next on my schedule.
14:55:39 <karsten> iwakeh: thanks!
14:55:56 <karsten> * metrics-web Jetty refactoring progress (iwakeh)
14:56:00 <irl_work> wait
14:56:03 <karsten> yes?
14:56:19 <irl_work> can we map the meanings of things in the ipv6 relay stats to relay search flags and consensus-health flags?
14:56:22 <irl_work> in their semantics?
14:56:38 <karsten> uhmmmm
14:57:15 <irl_work> i just don't want the glossary to explode
14:57:18 <karsten> not now, but yes.
14:57:20 <tjr> Please be very careful mapping anything to the IPv6 consensus-health flag, since I literally made up what it means based on what teor told me to do in a ticket. :)
14:57:43 <tjr> (Not that I think it's wrong, it just might not be what you expect it to be)
14:57:45 <karsten> let me find the specification patch.
14:57:59 <karsten> https://gitweb.torproject.org/karsten/metrics-web.git/commit/?h=tasks-24218-23761&id=57c58b5f1b099ac61e09169661a19b8c4a82984f
14:58:17 <karsten> that's the meaning for these new graphs. based on teor's suggestion.
14:59:04 <karsten> does that match what relay search and consensus health use?
14:59:30 <karsten> should we discuss this on a ticket?
14:59:36 <karsten> or email thread?
14:59:45 <irl_work> i think it matches
15:00:02 <irl_work> i'll look tonight and if very different then i'll start an email thread
15:00:06 <karsten> perfect!
15:00:19 <karsten> thanks for thinking of that!
15:01:09 <karsten> okay,
15:01:10 <karsten> * metrics-web Jetty refactoring progress (iwakeh)
15:01:25 <iwakeh> Now I have the changes restructured for ease of review
15:01:26 <karsten> I'm mainly asking, because I don't want to make this harder by merging things now.
15:01:32 <iwakeh> (30+ commits)
15:01:38 <karsten> cool!
15:01:43 <iwakeh> and as examples for refactoring of cureeently developed modules.
15:01:54 <iwakeh> The things causing trouble are edge cases that would also
15:02:01 <iwakeh> show up
15:02:09 <iwakeh> if one tried to set-up the old type anew, I think.
15:02:15 <karsten> quite possible.
15:02:22 <iwakeh> Is is very cumbersome to retrieve a partial dump of each
15:02:24 <karsten> so, this is ready for review now? or later today? or when?
15:02:31 <iwakeh> of the involved dbs?
15:02:49 <karsten> no, I can provide that.
15:03:13 <iwakeh> Review might start this week, worst case
15:03:27 <karsten> ok.
15:03:29 <iwakeh> is some minor questions are posted
15:04:03 <iwakeh> with the code.  But, with the db content it might be easier to test.
15:04:14 <karsten> will send you something later today.
15:04:21 <iwakeh> perfect!
15:04:38 <karsten> okay, I think we ran out of topics on the agenda.
15:04:50 <iwakeh> ah
15:05:07 <iwakeh> exonerator, I could provide a little patch for
15:05:14 <iwakeh> debugging & error catching.
15:05:25 <karsten> yes, please do.
15:05:40 <iwakeh> ok
15:06:03 <karsten> alright.
15:06:06 <karsten> anything else?
15:06:12 <iwakeh> I'm fine.
15:06:25 <irl_work> hiro: on op-ab, still waiting for new hardware so we may leave this until next year
15:06:36 <hiro> ok
15:06:46 <hiro> I did the changes regarding managed files
15:06:55 <hiro> it should work now ;) at least i tested it locally
15:06:58 <hiro> w vagrant
15:07:01 <irl_work> yay (:
15:07:16 <irl_work> thanks for doing that
15:07:24 <hiro> next year I shall have also prometheus stuff added to that
15:07:29 <hiro> I am working on that these days
15:07:35 <irl_work> ok cool (:
15:08:12 <irl_work> i have no other things for the meeting
15:08:23 * karsten ran out of topics, too.
15:08:39 <iwakeh> well, back to work :-)
15:08:56 <karsten> sounds good! thanks, everyone! :)
15:08:58 <karsten> bye, bye.
15:09:00 <irl_work> bye!
15:09:03 <karsten> #endmeeting