14:01:01 <karsten> #startmeeting metrics team
14:01:01 <MeetBot> Meeting started Thu Nov  3 14:01:01 2016 UTC.  The chair is karsten. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:01:01 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
14:01:06 <karsten> hi linda!
14:01:13 <linda> hello, karsten!
14:01:41 * karsten finds the agenda pad..
14:01:55 * linda waits patiently ♫♩♪
14:01:59 <karsten> https://pad.riseup.net/p/3M7VyrTVgjlF
14:02:20 * linda goes to pad
14:02:59 * karsten created an item for ux things.
14:03:09 <linda> Yay! And it's pink~
14:03:16 <karsten> hehe
14:03:22 <linda> is iwakeh here?
14:03:40 <linda> And how can I best contribute? (be a fly on the wall | talk a lot | etc)
14:04:00 <karsten> so, usually we spend the first few minutes to write agenda items.
14:04:12 <linda> Oh, I see.
14:04:18 <karsten> let me quickly put an item or two there.
14:04:21 * linda admires agenda pad. UX should do something like this.
14:04:42 <iwakeh> hi there!
14:04:51 <karsten> hi iwakeh!
14:04:58 <hiro> hi!
14:05:04 <linda> ヽ(•◡•)ﾉ
14:05:08 <karsten> hi hiro!
14:05:51 * karsten put linda's and hiro's items at the top, so that they can decide whether they want to stay around the full 45 minutes.
14:06:07 <karsten> in fact, let me switch them, hiro's should be quickest.
14:06:10 <linda> how considerate. (◕o◕)
14:06:10 <iwakeh> that why I put the dotted item on top :-)
14:06:13 * linda is okay with this
14:06:37 <karsten> okay, should we start?
14:06:42 <iwakeh> sure.
14:06:46 <linda> (~˘▾˘)~
14:06:49 <hiro> I am ok to stay, since I am learning I might get to know what you will be doing in the upcoming weeks
14:06:59 <karsten> great!
14:07:02 <karsten> * pull request for INSTALL.md (hiro)
14:07:16 <karsten> so, you set up collector and have suggestions for making the manual even better! :)
14:07:26 <iwakeh> great!
14:07:31 <karsten> let's see what we have on the team faq for pull requests...
14:08:00 <karsten> https://trac.torproject.org/projects/tor/wiki/org/teams/MetricsTeam/Documentation#HowdoIsubmitapatch
14:08:23 <karsten> I think that's a good answer.
14:08:33 <karsten> does it make sense, hiro?
14:08:41 <hiro> yep
14:09:04 <karsten> cool! looking forward to the patch. thanks for working on that!
14:09:21 <karsten> moving on:
14:09:26 <karsten> * UX project and Berlin preparations (linda)
14:09:38 <linda> ᕦ(^o^)ᕤ
14:09:47 <karsten> so, you wrote a pretty neat project page. /me finds the link...
14:09:55 <linda> https://trac.torproject.org/projects/tor/wiki/doc/UX/MetricsRedesign
14:10:05 <karsten> right, that. :)
14:10:26 <karsten> I guess my main question is: what can iwakeh and I do to be most useful next week?
14:10:31 * linda plans to finish all the analyses by today and add it to the page.
14:10:59 <linda> I think that the best ways to help are
14:11:26 <linda> 1) come up with ideas on how to improve the metrics page so we can discuss (but it seems like you did that already /refers to your doc)
14:11:53 <iwakeh> and we're sort of a little blind according to usability ;-)
14:11:55 <linda> 2) help me to understand what your vision for the metrics page specifically is, so that I can better help.
14:12:22 <linda> I love usability and have studied it, but I feel that I can be more useful if I knew more.
14:12:28 <karsten> hmm, vision with regard to what?
14:12:46 <iwakeh> maybe main target group?
14:12:49 <linda> I don't know common complaints about the metrics page, how many peope use it, if you have plans to try to add more functionality or to make the existing stuff better, etc.
14:13:08 <linda> I feel like all of this is stateful information that you two specifically have.
14:13:27 <linda> some specific questions:
14:13:52 <linda> -is the metrics page used by law enforcement people or people who are affected by Tor users? or should we refer to blogs or the Tor website for this?
14:14:09 <linda> -what are some of the "visuals that people requested" that is referred to in the doc?
14:14:21 <linda> -what parts of the page do you really like, and what parts do you not like?
14:14:27 * linda ends firehose of questions
14:14:39 <karsten> great questions. I guess we can guess about the answers.
14:14:48 <karsten> but we might even ask people in the remaining week until berlin.
14:14:56 <linda> That sounds good!
14:15:33 <linda> I'm also open to investigating what it would look like to cater specifically for certain user groups (i.e. researchers, journalists, people who are affected by Tor), and then decide that it's nto worth pursuing/for later.
14:15:33 <iwakeh> I might be also easier to look at the pages together and talk.
14:15:41 <linda> I agree.
14:16:08 <iwakeh> right, and technical feasibility.
14:16:12 <linda> mmmhmm
14:16:18 <iwakeh> hi ;-)
14:16:41 <iwakeh> when having to decide between features.
14:16:50 <linda> I think my thoughts before our meeting was "omg karsten has put a lot of thought into this and I want to catch up. And I want to do as much work I can independently before we meet as possible."
14:17:41 <linda> I can't think of specific things you two would need to do (although things like showing me the problems you identified, your user types, etc. were helpful but I didn't think to ask because I didn't knwo they existed)
14:18:07 <iwakeh> yep, that should lead to the best result when we combine all.
14:18:14 <linda> :)
14:18:22 <linda> My ideas for the meeting are:
14:18:36 <linda> -walk linda through the metrics page and tell linda what you guys like/don't like
14:18:43 <linda> -tell me all of your ideas and goals
14:19:03 <linda> -I share my usability inspection and Isa's best practices recommendations
14:19:18 <linda> -we brainstorm ideas and draw a bunch of stuff to see what we like
14:20:26 <linda> I'm open to other things to do!
14:20:34 <karsten> oh, just comes to mind: we do have statistics on which parts of the website are most popular.
14:20:45 <linda> ! ♡
14:21:39 <karsten> and regarding the question above how law enforcement people find things, that's a more general question how anybody finds the metrics website or services like exonerator at all.
14:21:45 <iwakeh> does metrics have a facebook, twitter etc.? As mentioned in the wiki page?
14:21:51 <karsten> we should think about integrating it/linking it from the tor website.
14:22:19 <karsten> ah, I think that's "how facebook does metrics".
14:22:30 <iwakeh> ah, ok.
14:22:53 <iwakeh> but aren't that different things?
14:23:01 <linda> Hmm.
14:23:11 <linda> I'm curious as to how people find most things.
14:23:13 <iwakeh> well, nevermind, just wait for the result.
14:23:27 <iwakeh> we have the
14:23:34 <karsten> iwakeh: I'm very curious how big companies do metrics things.
14:23:55 <linda> iwakeh: isabela is looking at google's, facebook's, and twitter's metrics pages!
14:24:01 <iwakeh> difficulty of having to explain a complex content on the metrics pages.
14:24:24 <linda> iwakeh: and she is making a pdf summary and giving us her recommendations. She also worked at twitter so I think she has some knowledge of how they did it internally.
14:24:41 <iwakeh> that's cool.
14:24:45 <linda> Yeah, we explain less common concepts.
14:25:00 <linda> And I suspect that their way is "we have an offical metrics team of 20 pepople, that's how we do it."
14:25:04 <linda> but anyway.
14:25:08 <karsten> right.
14:25:46 <linda> karsten: could you send me the statistic for the most loved metrics things?
14:25:47 <karsten> other limitations we have is that we're trying hard to avoid javascript and that we want to make everything run on debian stable.
14:25:55 <karsten> linda: yes, will do!
14:26:01 <linda> karsten: yay.
14:26:09 <linda> Ah, I see. that's good to know.
14:26:43 <iwakeh> and we don't have the resources to maintain the page layout very often.
14:27:01 <linda> totally understandable
14:27:05 * linda is a team of 1.
14:27:15 * linda is supposedly UX team lead.. is leading herself?...
14:27:20 <karsten> :)
14:27:34 <linda> Alright.
14:27:50 <linda> for clarity, are you both okay with my proposed things to do f2f?
14:27:59 <karsten> sounds good to me!
14:28:01 <linda> (and after this I have one more question~)
14:28:03 <linda> yay!
14:28:07 <iwakeh> sure.
14:28:12 <linda> How thorough of a usability analysis would you like for the page? I know it's a deliverable for Sponsor X.
14:28:24 <linda> I can always do more analyses, until I did "enough" (although you just work endlessly if you do this, and I argue it's not too productive to test exactly how unusable it is if you know it is unusable and spending time trying to fix it is more fruitful). But if it helps the sponsor, I could do user testing, user surveys, or record things ('i.e. for me to find X, it took me 3 clicks and 4 inputs').
14:28:36 <iwakeh> I think we should define that in Berlin.
14:28:43 <linda> I planned to do a rough evaluation (meets needs/does not) before the Berlin meeting.
14:28:48 <linda> Oh, that makes sense.
14:28:53 <iwakeh> It's easier to describe the levels etc.
14:29:07 <linda> We can even have comparisons before and after, if that is helpful.
14:29:19 <karsten> so, the text on the SponsorX page is what we said we'd do.
14:29:23 <linda> "i.e. before, it required <blah> but now it is only <blah>"
14:29:26 <karsten> we didn't give more details what exactly that will be.
14:29:39 <karsten> so, the primary goal should be to produce something that's useful to us.
14:29:48 <linda> Okay.
14:30:10 <linda> Future funding is also useful. ;)
14:30:19 <iwakeh> always :-)
14:30:39 <linda> But yeah, let's decide that at the meeting. I'll update the wiki page so we remember.
14:30:41 <karsten> and to be clear, we won't be able to implement all the things now, also for technical reasons.
14:30:48 <linda> Oh of course.
14:30:54 <karsten> I know that more interactive graphs would be nice to have.
14:31:02 <karsten> will probably not happen this year.
14:31:05 <linda> That sounds painful to do correctly though.
14:31:05 <iwakeh> so prioritization will be good from the start.
14:31:25 <linda> I agree. We should brainstorm big, then prioritize, then pick off the feasible stuff.
14:31:31 <karsten> +1
14:31:36 <iwakeh> yep.
14:31:43 <linda> I really liked your dashboard idea, karsten.
14:31:54 <karsten> ah, that was iwakeh's, I think. :)
14:32:00 <iwakeh> yep :-)
14:32:06 <linda> And something I've anecdotally (n=1) heard was it would be nice to see everything about a country.
14:32:09 <karsten> I'd be too worried about technical feasibility to suggest such a thing. ;)
14:32:15 <linda> iwakeh: /highfive
14:32:26 <linda> karsten: which is good to have in balance to iwakeh ~
14:32:32 <karsten> speaking of, we co-authored that doc with suggestions.
14:32:43 <karsten> not sure if it matters much.
14:32:50 <linda> Oh, I shall reflect the wiki to give credit where credit is due.
14:32:56 <karsten> ok.
14:32:57 <linda> Not sure either, but nice to know.
14:33:14 <karsten> regarding the dashboard,
14:33:24 <karsten> I'd love to make it easier for people to customize their use of metrics.
14:33:30 <karsten> and even define their own metrics based on our data.
14:33:41 <linda> ☝ this
14:33:43 <karsten> or at least make it really easy for us to add new stuff based on suggestions.
14:33:48 <linda> mmhmm!
14:33:50 <karsten> right now it takes lots of code.
14:33:53 <linda> Yeah.
14:33:54 <linda> :(
14:34:05 <iwakeh> this would be a candidate for future direction, vision.
14:34:10 <linda> Something I would also like is for people to somehow annotate the graphs..
14:34:18 <iwakeh> tagging?
14:34:19 <karsten> yes!
14:34:29 <linda> I know isabela has specific knowledge about weirdness in Brazil related to a botnet attack that used Tor.
14:34:34 <iwakeh> you mean incident tagging?
14:34:38 <linda> And I know dcf has a not of meek-specific stuff
14:34:40 <linda> Oh yeah
14:34:52 <karsten> we have something like that in the doc. that would be really nice.
14:34:57 <karsten> and maybe even doable.
14:34:58 <linda> also quite hard.
14:35:07 <iwakeh> well it depends ..
14:35:14 <linda> and we should make it clear that we don't necessarily claim it to be TRUE TRUE TRUE.
14:35:18 <linda> Oh hehe
14:35:42 * linda maybe was imagining something really fancy, like having bubbles pop up next to spikes/dips in graphs
14:35:48 <iwakeh> just metrics, who talks about truth.
14:36:16 <karsten> well, we could start with a table below the graph or something.
14:36:16 <linda> (≧◡≦)
14:36:33 <karsten> anyway, we should probably go through all these ideas next week.
14:36:35 * linda commends karsten for practicality
14:36:38 <linda> yes yes
14:36:43 <linda> we're getting ahead of ourselves.
14:36:49 <linda> I've gotten my questions answered.
14:36:53 <karsten> heh. great!
14:37:10 <karsten> and I have a bunch of questions to think about more while sitting on a bus next week.
14:37:14 <linda> I don't think I would like anything from you guys, except maybe a plan of walking me through, and more ideas if you have any more in addition to the report.
14:37:35 <iwakeh> fine.
14:37:38 <karsten> sounds good.
14:37:50 <karsten> alright, shall be move on?
14:37:55 <linda> I'll do a rough evaluation, gathe isa's intel, and come up with as many ideas as I can.
14:37:58 <linda> mmhmm! I'm done~
14:38:06 <karsten> very curious!
14:38:08 <linda> Thanks for letting me sit in. :3
14:38:16 <karsten> thanks for being here. :)
14:38:17 <linda> (~˘▾˘)~
14:38:23 <karsten> * CollecTor: convert ideas into tasks (iwakeh)
14:38:30 <iwakeh> right,
14:38:44 <iwakeh> I'm tidying up the lose ends and
14:39:01 <iwakeh> convert ideas into trac tickets for future reference and work.
14:39:18 <karsten> yes, I noticed new tickets. :)
14:39:28 <iwakeh> (sorry for the 'spam')
14:39:33 <karsten> ah, no worries.
14:39:46 <iwakeh> some miight already turn into
14:39:56 <iwakeh> metrics-help & hack-day
14:40:00 <iwakeh> tickets.
14:40:06 <karsten> yep.
14:40:15 <iwakeh> I put some on needs_info soon
14:40:29 <karsten> though I'm not yet sure how many java people will come to that.
14:40:29 <iwakeh> when the topic needs discussion or input.
14:40:39 <karsten> okay, should I look out for needs_info then?
14:40:39 <iwakeh> oh, well
14:40:50 <iwakeh> yes :-)
14:40:56 <karsten> okay.
14:41:02 <iwakeh> It's actually
14:41:20 <iwakeh> intended to turn the bigger tasks into
14:41:32 <iwakeh> small pieces we can finish here and there
14:41:39 <iwakeh> and eventually put together.
14:41:50 <karsten> sounds good.
14:42:15 <iwakeh> So, do we need more for the hack-day event?
14:42:23 <karsten> I have one idea.
14:42:27 <karsten> should move to that topic?
14:42:31 <karsten> should we*
14:42:40 <iwakeh> sure.
14:42:43 <karsten> * metrics-help & hack-day (iwakeh)
14:42:55 <karsten> there will be ooni people, and we had plans a while back to put ooni visualizations on metrics.
14:43:04 <karsten> let's work on that while ooni people are around.
14:43:13 <iwakeh> ah, sound great!
14:43:18 <karsten> which creates one question I don't have a good answer to:
14:43:36 <karsten> right now, we're only visualizing things on metrics that are based on data from collector.
14:43:49 <iwakeh> th external data question?
14:43:51 <karsten> should we give up on that and have metrics fetch data from ooni?
14:43:56 <karsten> yes, that question.
14:44:03 <karsten> or should we include ooni data in collector?
14:44:18 <iwakeh> hmm, what else would that imply?
14:44:42 <karsten> well, we'll ask us the same question when adding onionperf data.
14:44:46 <iwakeh> rather collect at collector to have one source.
14:45:10 <karsten> and at some point we'll want to think about the future of webstats.tp.o.
14:45:20 <karsten> and whether or not to make it a collector module.
14:45:22 <iwakeh> I'm not really familiar with the data gathering storage for ooni?
14:45:28 <karsten> oh, me neither.
14:45:50 <karsten> but we can find out next week.
14:45:59 <iwakeh> I think collector might be a good place for
14:46:23 <iwakeh> all that data; especially for archiving data.
14:46:45 <iwakeh> next week is fine. team meeting f2f then?
14:46:49 <karsten> here's another one: geoip databases, data about world population (if we ever want to use that), etc.
14:47:05 <karsten> next week sounds great!
14:47:11 <karsten> as you see I don't have a good answer.
14:47:13 <iwakeh> yes, didn't we plan that already?
14:47:21 <iwakeh> the geoip.
14:47:26 <karsten> did we?
14:47:38 <karsten> ah, I'm not talking about onionoo.
14:47:39 <iwakeh> which leads to Onionoo.
14:47:47 <iwakeh> ah, ok.
14:47:48 <karsten> no, sorry, I didn't mean that.
14:48:07 <karsten> what if we want to integrate geoip data in metrics graphs, like for reviving the relays-by-country graph.
14:48:07 <iwakeh> you mean gḿore general data?
14:48:35 <karsten> (which was killed many years ago..)
14:48:52 <iwakeh> why?
14:49:16 <karsten> uhhmmm...
14:49:21 <karsten> slow geoip in psql?
14:49:33 <karsten> don't recall exactly, but no reason with much relevance today.
14:49:37 <iwakeh> technical reasons?
14:49:40 <karsten> yep.
14:49:53 <iwakeh> let's re-visit that topic.
14:49:55 <hiro> what if you have few relays in an area?
14:50:11 <iwakeh> dangerous for privacy.
14:50:16 <hiro> yeah that
14:50:28 <karsten> wait, relays.
14:50:35 <karsten> those are public.
14:50:50 <hiro> ah ok right
14:51:18 <karsten> okay, moving on.
14:51:22 <iwakeh> sure.
14:51:24 <karsten> * Onionoo next steps (iwakeh)
14:51:38 <karsten> so, we briefly talked about that last week or the week before.
14:51:45 <karsten> with the plan to keep the release small.
14:51:50 <iwakeh> yep, my task list
14:52:16 <iwakeh> contains the planning topic. which
14:52:44 <iwakeh> should be done tomorrow, the draft.
14:53:04 <iwakeh> are the very important tickets set to high?
14:53:13 <karsten> uh, maybe.
14:53:14 <iwakeh> important for others?
14:53:28 <karsten> I wouldn't rely on that.
14:53:50 <iwakeh> ok. i keep that in mind.
14:53:52 <karsten> so, I see a lot of important work planned for november.
14:54:16 <karsten> I wonder if we should postpone onionoo release and manual. hmm.
14:54:29 <iwakeh> why?
14:54:30 <karsten> after all, we cannot complete that task without atlas improvements anyway.
14:55:11 <iwakeh> switching the tor-daemon analysis for onionoo?
14:55:14 <karsten> I mean, we could put out a small release now and consider doing more when we're working more on atlas.
14:55:22 <karsten> oh, right.
14:55:38 <iwakeh> a small release would be doable and
14:55:49 <iwakeh> it's better to release fast and often.
14:55:58 <karsten> yep.
14:56:02 <karsten> okay.
14:56:13 <iwakeh> so, I have that in mind too
14:56:20 <iwakeh> when planning
14:56:33 <iwakeh> the two to three releases ;-)
14:56:55 <karsten> hmm?
14:57:16 <iwakeh> There are many onionoo tickets.
14:57:21 <karsten> yeah.
14:57:50 <iwakeh> how much time to reserve for the oninoo release?
14:58:10 <karsten> including the manual? days?
14:58:31 <iwakeh> rather aiming at release date.
14:59:01 <karsten> the release date should be after berlin.
14:59:06 <iwakeh> the manual should be releasable in Nov '16.
14:59:13 <karsten> oh, yes.
14:59:37 <karsten> ping a random date: nov 23.
14:59:47 <karsten> picking*
14:59:52 <karsten> sorry, random date.
15:00:05 <iwakeh> 23 is fine
15:00:20 <iwakeh> not very random prime ;)
15:00:40 <karsten> yeah, not really random.
15:00:52 * linda lurks and giggles
15:00:57 <karsten> oh, 60 minutes into the 45 minute meeting.
15:01:01 <karsten> moving on?
15:01:20 <karsten> * trac: ticket info mails for Metrics projects (iwakeh)
15:01:34 <iwakeh> yep.
15:01:48 <iwakeh> you did have a suggestion?
15:01:59 <karsten> well, I wrote how the tbb team does this.
15:02:04 <karsten> or how I think they do it.
15:02:19 <karsten> they have a user tbb-team or something that is the component owner.
15:02:29 <karsten> and that gets mail to a mailing list.
15:02:42 <iwakeh> should we just open an admin-ticket?
15:03:11 <karsten> I'm not sure if there are drawbacks.
15:03:23 <karsten> maybe we should ask them first how this is working for them.
15:03:40 <iwakeh> sure.
15:04:09 <karsten> okay!
15:04:13 <karsten> moving on to the last item:
15:04:14 <karsten> * handling bad input and common log levels (karsten)
15:04:21 <iwakeh> noticed the tickets.
15:04:22 <karsten> I opened that ticket today. finally.
15:04:44 <karsten> I feel that making progress here would be quite useful for the various code bases.
15:05:17 <iwakeh> yes, and it can be part of the first onionoo rel.
15:05:28 <iwakeh> the logging stuff.
15:05:38 <karsten> sure!
15:06:25 <karsten> what's the logging ticket again?
15:06:43 <iwakeh> #20340
15:06:51 <iwakeh> #20540
15:07:13 <karsten> okay, will move that forward.
15:07:14 <iwakeh> #20540 is the parent for general discussion.
15:07:16 <karsten> by replying.
15:07:21 <iwakeh> yep.
15:07:36 <karsten> maybe take a look at #20548 in the meantime?
15:07:55 <iwakeh> skimmed through already.
15:08:46 <karsten> okay!
15:09:02 <karsten> does that mean we're done for today?
15:09:18 <iwakeh> yep.
15:09:35 <karsten> lots of topics, good meeting!
15:09:40 <iwakeh> right!
15:09:41 <hiro> yep
15:09:50 <karsten> so, no (IRC) meeting next week.
15:09:58 <iwakeh> back to work! :-)
15:10:08 <karsten> quick quick!! ;)
15:10:13 <iwakeh> yes
15:10:17 <karsten> thanks, iwakeh, hiro, linda!
15:10:25 <karsten> #endmeeting