14:32:25 <karsten> #startmeeting metrics team
14:32:25 <MeetBot> Meeting started Thu Nov  9 14:32:25 2017 UTC.  The chair is karsten. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:32:25 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
14:32:32 <karsten> * Jetty switch and JSP and dependency fun (iwakeh)
14:32:43 <iwakeh> getting there
14:32:52 <iwakeh> exonerator works and now
14:33:18 <iwakeh> I'm testing and working on 24175
14:33:21 <iwakeh> #24175
14:33:37 <karsten> is there already a patch for exonerator?
14:33:41 <iwakeh> to have the synergy for recognizing.
14:33:46 <karsten> ah ok.
14:33:48 <iwakeh> no, I want to wait
14:33:48 <karsten> makes sense.
14:33:54 <karsten> sounds good.
14:34:01 <iwakeh> to streamline both as metrics-web is more
14:34:12 <iwakeh> complex and some things might derive from it
14:34:21 <iwakeh> and added/changed in exonerator.
14:34:28 <karsten> ok.
14:34:37 <iwakeh> soon there
14:34:55 <iwakeh> will be the `java -jar exonerator.jar`
14:35:14 <iwakeh> and java -jar metrics-web.jar (with versions).:-)
14:35:26 <karsten> neat!
14:35:28 <iwakeh> ah, a question
14:35:32 <karsten> yes :)
14:36:03 <iwakeh> as m-web has many modules these will be in one jar (restructured to use metrics-base)
14:36:25 <iwakeh> Does it suffice to call
14:36:52 <iwakeh> java -cp metrics-web.jar <org.t.some mainclass>
14:37:01 <iwakeh> for running these?
14:37:12 <karsten> you mean right now?
14:37:21 <iwakeh> or would you like the scripts or a commend line arg?
14:37:34 <iwakeh> in general?
14:37:40 <karsten> in some cases we still need the scripts.
14:37:40 <iwakeh> rather
14:37:46 <iwakeh> ok
14:37:50 <karsten> because we're doing non-java things.
14:37:54 <iwakeh> that's true
14:37:57 <karsten> fixing that is a longer-term thing.
14:38:06 <iwakeh> then adapt these scripts accordingly. fine.
14:38:07 <karsten> worthwhile, but probably not as part of this change.
14:38:11 <karsten> yes.
14:38:29 <karsten> while we're talking about metrics-web... :) next topic?
14:38:37 <iwakeh> ok
14:38:42 <karsten> * IPv6 graphs: specification and implementation (karsten)
14:38:56 <karsten> so, teor and I discussed possible ipv6 graphs on a ticket.
14:39:03 <karsten> and I think we have two good candidates.
14:39:08 <iwakeh> nice.
14:39:36 <karsten> plus we'll probably have another one or two after isis says what they could imagine for bridges.
14:39:51 <karsten> however, two questions for today:
14:40:16 <karsten> 1. we'll have to cover the new CSV file in the documentation for sponsor 13.
14:40:22 <iwakeh> ok
14:40:35 <karsten> would it make sense to work on the documentation/specification while working on the code?
14:40:43 <iwakeh> more work though.
14:40:50 <karsten> overall? not sure.
14:40:59 <karsten> the alternative is to do it afterwards.
14:41:18 <karsten> I'm not sure.
14:41:20 <iwakeh> hmm.
14:41:24 <karsten> something to consider.
14:41:44 <iwakeh> true, need to think more; actually, the spec should be first ;-)
14:42:08 <iwakeh> but, derive the description from code will work.
14:42:22 <iwakeh> maybe, keep a list of major decisions ?
14:42:23 <karsten> and 2. we'll need to write this new metrics-web module.
14:42:29 <karsten> re 1:
14:42:38 <karsten> I wrote the prototype code and included some comments.
14:43:08 <iwakeh> also questions/answers to why what was included in the graphs.
14:43:09 <karsten> but now my memory is still fresh, so I could write better documentation now than in a few weeks. I think.
14:43:18 <karsten> yes, and that.
14:43:34 <iwakeh> true.
14:43:40 <karsten> knowing a useful format for all this would help.
14:43:49 <iwakeh> hehe, I understand.
14:43:55 <karsten> so, re 2:
14:44:05 <iwakeh> wait
14:44:24 <iwakeh> re1: did we decide on LaTeX or txt?
14:44:25 <karsten> my current plan, which may not be the final plan, is to write a postgresql database importer and a database schema that does the aggregation.
14:44:31 <karsten> re 1: not at all.
14:44:40 <karsten> or, not that I'm aware of.
14:44:46 <karsten> or html?
14:44:54 <iwakeh> ergh
14:44:59 <karsten> or xml?
14:45:03 <karsten> or markdown?
14:45:19 <iwakeh> Rather LaTeX as we need formulars here nd there.
14:45:39 <iwakeh> there is latex2html.
14:45:47 <karsten> does that work for formulas?
14:46:17 <karsten> sounds like we should prototype this, too.
14:46:19 <iwakeh> there is the mathjax extension.
14:46:25 <iwakeh> yep.
14:47:35 <karsten> ok.
14:47:40 <karsten> re 2:
14:47:49 <karsten> my suggestion above is probably quite fast.
14:47:51 <iwakeh> restructuring
14:47:59 <karsten> but that's just for the moment.
14:48:14 <karsten> if we have to rewrite things shortly after, then it'll take more time overall.
14:48:33 <iwakeh> ? more time?
14:48:35 <karsten> and I believe we're not under huge pressure here.
14:48:41 <karsten> more dev time.
14:48:43 <iwakeh> shortly after what task?
14:48:53 <karsten> so,
14:49:14 <karsten> if we decide to switch to another database or aggregation method, we might have to rewrite the postgresql stuff.
14:49:22 <karsten> that would be in 2018.
14:49:33 <karsten> just saying, we can build something really quickly now,
14:49:50 <iwakeh> might be a nice achievement.
14:49:51 <karsten> or we can take some more time and build something that we'll more likely keep for the next years.
14:50:15 <karsten> I don't feel strongly here, I'm just mentioning alternatives.
14:50:15 <iwakeh> hmm, it should be 'likeable' from the beginning.
14:50:54 <iwakeh> Maybe, I can look at the code?
14:51:08 <karsten> the prototype is misleading in that regard.
14:51:14 <iwakeh> to get a feeling for how much it is.
14:51:16 <karsten> but I can write down my idea with more words.
14:51:21 <iwakeh> ah, ok.
14:51:45 <karsten> 350 lines of code. :)
14:52:09 <iwakeh> well, int x=0;
14:52:14 <iwakeh> int j=2;
14:52:20 <iwakeh> and and and ;-)
14:52:33 <karsten> and think of all the imports!!
14:52:38 <karsten> okay.
14:52:40 <iwakeh> R, java, SQL?
14:52:48 <iwakeh> all.
14:53:08 <karsten> the prototype doesn't use sql.
14:53:11 <karsten> just r and java.
14:53:28 <karsten> 20 lines of R, 350 lines of java.
14:53:40 <karsten> anyway, I think we can move on.
14:53:47 <iwakeh> ah, that will be additional dev-time, the db-code.
14:53:52 <iwakeh> ok
14:54:00 <karsten> yes, this prototype does not scale.
14:54:16 <karsten> ah!
14:54:21 <karsten> you thought the module is almost ready?
14:54:26 <karsten> not at all.
14:54:48 <iwakeh> ok
14:54:56 <karsten> so, next topic?
14:55:21 <iwakeh> fine.
14:55:25 <karsten> * Roadmap additions (karsten)
14:55:33 <karsten> did you see my mail from monday?
14:55:59 <karsten> where I summarized the inter-team meeting?
14:56:00 <iwakeh> I don't memorize when I received certain mails.
14:56:10 <iwakeh> ah, sure.
14:56:33 <iwakeh> that looked all ok.
14:56:39 <karsten> great!
14:56:49 <karsten> in that case I shall update the roadmap and send you and irl a new version.
14:56:57 <iwakeh> great.
14:57:05 <karsten> which will hopefully be the final version. or close to it.
14:57:21 <iwakeh> should be as we're on the road by now.
14:57:33 <karsten> yep.
14:57:53 <karsten> that's all for this topic.
14:58:00 <karsten> moving on:
14:58:02 <karsten> * Metrics timeline underneath graphs mockups (karsten)
14:58:31 <karsten> that's some recent work I did today.
14:58:40 <karsten> but I thought it would be useful to have some quick feedback.
14:58:50 <karsten> https://people.torproject.org/~karsten/volatile/metrics-news-mockup-1/userstats-relay-country-table.html
14:58:54 <karsten> https://people.torproject.org/~karsten/volatile/metrics-news-mockup-1/userstats-relay-country-text.html
14:58:58 <karsten> https://people.torproject.org/~karsten/volatile/metrics-news-mockup-1/userstats-relay-country-indented.html
14:59:31 <karsten> look at the "Possibly related events" section.
14:59:43 <iwakeh> nice,
14:59:46 <karsten> any quick thoughts on which version works better and why?
15:00:00 <karsten> or what else I should try?
15:00:07 <karsten> (it's all HTML hacking so far.)
15:00:38 <karsten> feel free to respond on the list if you'd rather take some more time.
15:00:46 <iwakeh> the indented text is
15:00:58 <iwakeh> most readable to me.
15:01:10 <karsten> cool!
15:01:29 <iwakeh> the table takes up too much space from the text for the tags and doesn't fit on screen.
15:01:51 <iwakeh> non-indented text is harder too read.
15:02:11 <karsten> yeah.
15:02:27 <iwakeh> the items listed will be marked
15:02:27 <karsten> what about the table lines? are they useful?
15:02:34 <iwakeh> on the graph?
15:02:38 <karsten> later, yes.
15:02:47 <iwakeh> fine.
15:02:56 <karsten> and there will be fewer entries on other graphs, I think.
15:03:04 <karsten> well, for other 3-month graphs.
15:03:34 <iwakeh> there could be an adkustable limit?
15:03:39 <iwakeh> adjustable
15:03:47 <karsten> tricky.
15:03:50 <iwakeh> for items.
15:03:57 <karsten> making it adjustable for the user means adding another parameter.
15:04:09 <iwakeh> yes.
15:04:10 <karsten> setting a limit in the code means possibly removing the most relevant items.
15:04:34 <iwakeh> no, it should be a user adjust. parameter.
15:04:34 <karsten> maybe there's js magic to hide items.
15:04:49 <iwakeh> something like that would be fine.
15:04:55 <karsten> like, show the first 3 or 4 and a button for "more".
15:05:10 <karsten> which is totally not something I could build. but maybe we'll get help with that.
15:05:36 <iwakeh> web-magician's help :-)
15:05:38 <karsten> regarding table lines, do you feel that it would help to add lines between items in the indented version?
15:05:44 <karsten> without table header, though.
15:06:02 <karsten> I guess I can try it out as version 4.
15:06:03 <iwakeh> no line necessary.
15:06:19 <iwakeh> for me. I might not have the popular taste here.
15:06:31 <karsten> I'm usually against adding lines, too.
15:06:39 <karsten> same applies regarding popular taste.
15:06:47 <karsten> alright. thanks, that's some great input!
15:06:48 <iwakeh> I could imagine that ppl would like mouse over
15:06:54 <iwakeh> info on the graph.
15:07:18 <karsten> yeah, that's step 3.
15:07:28 <iwakeh> or a link.
15:07:29 <karsten> with this being step 1 and visual annotation in the graph being step 2.
15:07:32 <iwakeh> ok
15:07:44 <iwakeh> a good start!
15:08:00 <iwakeh> the event lict is helpful.
15:08:07 <karsten> I'm somewhat worried about putting too much effort into this now and later getting it for free from a visualization framework.
15:08:14 <karsten> yes, I think it's a start.
15:08:26 <iwakeh> no risk, no fun :-)
15:08:49 <iwakeh> I think frameworks will need adaption and work.
15:09:12 <iwakeh> so, having defined what looks good/usable will be useful anyway.
15:09:33 <karsten> true.
15:10:06 <iwakeh> I would change the table title.
15:10:12 <karsten> yes?
15:10:28 <iwakeh> "Possibly related events" -> "Events during this time"
15:10:39 <iwakeh> to avoid correlation suggestions.
15:10:47 <karsten> makes sense.
15:11:04 <iwakeh> Or just "Events"
15:11:12 <iwakeh> and the dates.
15:11:26 <karsten> so, these are not all events.
15:11:42 <karsten> if the user looks at a graph for Turkey, they won't see events from France.
15:11:46 <iwakeh> Which are missing?
15:11:53 <iwakeh> ah.
15:12:20 <iwakeh> Events from the chosen country and general events?
15:12:50 <iwakeh> Maybe, make this another choice (in version 4)?
15:12:52 <karsten> and the by-transport and by-version graphs will contain a different subset.
15:13:14 <karsten> and we could make different choices for relay users and bridge users.
15:13:24 <karsten> I mean, we don't have to.
15:13:28 <karsten> what's most useful? :)
15:13:41 <karsten> in some cases, a bridge-related event might be relevant for relay usage.
15:13:56 <karsten> and an even in France might affect users in Turkey. maybe.
15:14:02 <iwakeh> right.
15:14:17 <iwakeh> therefore have the choice
15:14:27 <karsten> choice?
15:14:36 <iwakeh> "show all" check-box
15:14:43 <karsten> ah! js.
15:14:48 <iwakeh> or something better.
15:15:04 <karsten> let's put it on the open questions and make a decision.
15:15:32 <iwakeh> ok.
15:16:00 <karsten> okay! 1 topic left.
15:16:05 <karsten> * metrics-2017 tickets (karsten)
15:16:15 <karsten> we already worked off a lot of them.
15:16:42 <karsten> https://trac.torproject.org/projects/tor/wiki/doc/Metrics2017Tickets
15:16:44 <iwakeh> yep.
15:16:55 <karsten> so, regarding the format,
15:17:14 <karsten> it's a bit sad that we need to use a wiki page for this.
15:17:25 <iwakeh> as opposed to?
15:17:32 <karsten> but I personally find the next steps information quite useful.
15:17:46 <karsten> so, what we could do is have another field in trac for next steps.
15:17:48 <karsten> just a line.
15:18:03 <karsten> and then include that field in the automatically generated ticket table below.
15:18:09 <iwakeh> or, just add a line to the description?
15:18:11 <karsten> you know, use it like a database.
15:18:13 <iwakeh> for a start?
15:18:22 <iwakeh> ah, ok.
15:18:37 <karsten> we need something that we can include in the table.
15:18:43 <iwakeh> I thought, your question was just about avoiding
15:19:01 <iwakeh> to read through zillions of comments.
15:19:06 <karsten> yes.
15:19:08 <iwakeh> yep.
15:19:12 <karsten> summarize where the ticket stands.
15:19:32 <iwakeh> some things can be done via tags.
15:20:05 <karsten> hmmmm
15:20:21 <iwakeh> like the tickets being worked on, e.g., the jetty tickets I would tag as "under-construction" .
15:20:30 <iwakeh> Better wording necessary.
15:20:53 <iwakeh> That could mean it is worked on, w/o any blocks etc.
15:21:05 <karsten> so, conventions for using other fields.
15:21:09 <karsten> including status.
15:21:31 <karsten> maybe.
15:21:31 <iwakeh> would be achievable already.
15:22:01 <karsten> okay, I'll think more about that.
15:22:04 <iwakeh> trac is stable now, every change might be troublesome.
15:22:17 <karsten> I think adding fields is relatively easy.
15:22:21 <karsten> as compared to adding plugins.
15:22:30 <iwakeh> valid point.
15:22:34 <karsten> and the other teams don't have to use these new fields.
15:22:53 <iwakeh> might complain about another field taking up space.
15:22:59 <karsten> yeah.
15:23:13 <karsten> I'll offer points and actual points in exchange. ;)
15:23:33 <karsten> alright.
15:23:50 <karsten> I think we covered a lot of topics.
15:23:55 <iwakeh> true.
15:23:55 <karsten> anything else?
15:24:12 <iwakeh> I'm fine, ready to look at dependencies again ;-)
15:24:27 <karsten> hehe. alright! thanks, and good luck with that! bye :)
15:24:36 <iwakeh> thanks, bye, bye!
15:24:41 <karsten> #endmeeting