14:29:43 <karsten> #startmeeting metrics team
14:29:43 <MeetBot> Meeting started Mon Nov 27 14:29:43 2017 UTC.  The chair is karsten. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:29:43 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
14:29:46 <karsten> hello!
14:29:50 <irl> hi
14:30:05 <karsten> hiro: around?
14:30:18 <hiro> yes here
14:30:26 <karsten> cool! shall we start?
14:30:31 <iwakeh> yes.
14:30:33 <hiro> ok
14:30:42 <karsten> * OnionPerf/Salt update (irl/hiro)
14:30:59 <irl> hiro: do you want to do this one?
14:31:03 <hiro> so we made some progress in that sense
14:31:05 <hiro> yep irl
14:31:11 <irl> cool
14:31:25 <hiro> the op instances: op-nl -hk -us are managed by salt
14:31:32 <hiro> the update script in salt is working
14:31:55 <hiro> so they are all running the same versions of the software at this time
14:32:01 <hiro> and can be upgraded and managed together
14:32:13 <karsten> is some of this under (public) version control somewhere?
14:32:30 <karsten> thinking of the monthly report and how to include this there.
14:32:31 <hiro> it is on oniongit at the moment
14:32:47 <hiro> the gitlab instance
14:32:55 <hiro> I can give you access
14:33:05 <karsten> can we include a link in the monthly report?
14:33:09 <hiro> there are no passwords so it is ok to share
14:33:23 <hiro> I have to make it public though I will share the link after this meeting
14:33:39 <karsten> sounds good!
14:33:55 <hiro> then with irl last week we made some test on how to bring his -ab instances to salt
14:34:05 <hiro> now he has also access to the salt master
14:34:28 <hiro> but there are a few things that I need to fix to make the provisioning scripts working
14:34:40 <hiro> the status is that as soon as I have those ready we will make more tests
14:34:49 <karsten> cool!
14:34:52 <hiro> should be this week
14:35:15 <irl> i'll be available for the testing but will need at least 1 day notice
14:35:19 <hiro> sure
14:35:49 <hiro> that's all from me on salt
14:36:04 <karsten> okay, and then we'll include op-ab in collector and on tor metrics?
14:36:15 <irl> that's the plan
14:36:19 <iwakeh> great!
14:36:58 <karsten> we might need 1 day for that, too, so if it happens until wednesday, we could include it in the november report, too.
14:37:12 <iwakeh> true.
14:37:15 <karsten> if not, december.
14:37:24 <irl> i'd like to wait adding op-ab, just for a week or so
14:37:27 <iwakeh> just a config change in CollecTor during runtime!
14:37:30 <karsten> hehe
14:37:32 <karsten> okay!
14:37:32 <irl> as the hardware it runs on may be being upgraded
14:37:40 <irl> and it won't be a zero downtime upgrade
14:37:58 <karsten> ok.
14:38:15 <karsten> next topic?
14:38:17 <irl> ok
14:38:25 <karsten> * Fallback directory list (irl)
14:38:35 <karsten> fine questions there.
14:38:41 <irl> so relay search now has a fallbackdir synthetic flag
14:39:00 <irl> this is generated by a hardcoded list in the javascript that is generated by a python script which pulls the list from gitweb
14:39:13 <karsten> ok.
14:39:13 <irl> every time this list is updated we need to just run the python script
14:39:26 <irl> but, stem doesn't have this issue, or other things using the list, because they have automatic updates
14:39:44 <irl> as relay search is static html/js it's not possible to automate it unless it's pulling it from a dynamic service
14:40:00 <irl> that's the context
14:40:15 <karsten> so, what we have right now sounds like a fine workaround for the moment.
14:40:16 <irl> so the two questions are can metrics-lib parse the fallbackdir file format, and should this be in onionoo?
14:40:39 <karsten> ideally, we'd add those files to collector, provide a parser in m-lib, and process those files in onionoo.
14:40:53 <irl> (for now, i'm being notified when it changes and this is in the procedure on the wiki page)
14:40:55 <iwakeh> sounds ok.
14:41:05 <irl> karsten: that sounds perfect
14:41:19 <karsten> the alternative, add it to onionoo only, is not so great.
14:41:27 <karsten> it's what we do with geoip files.
14:41:43 <karsten> and we need to coordinate updates. (btw, we should do one of those soon!)
14:41:52 <iwakeh> ok
14:42:01 <karsten> that's also why we're thinking about moving geoip files to collector.
14:42:10 <iwakeh> yep.
14:42:45 <karsten> so, the steps above will take some time.
14:43:04 <karsten> can somebody create a ticket?
14:43:10 <karsten> (or is there one already?)
14:43:11 <iwakeh> yes
14:43:18 <irl> the fallback list isn't updated that often, so this isn't super urgent
14:43:25 <iwakeh> I mean, I'll check and create if necessary ;-)
14:43:30 <irl> thanks iwakeh
14:43:31 <karsten> neat!
14:44:01 <iwakeh> @irl: which wiki page? for reference in the ticket?
14:44:20 <irl> https://trac.torproject.org/projects/tor/wiki/doc/FallbackDirectoryMirrors
14:44:25 <iwakeh> thanks!
14:44:33 <irl> sorry, not that one
14:44:34 <irl> https://trac.torproject.org/projects/tor/wiki/doc/UpdatingFallbackDirectoryMirrors
14:44:57 <iwakeh> ok
14:45:00 <karsten> alright, next?
14:45:03 <irl> ok
14:45:11 <karsten> * Updates to Relay Search (irl)
14:45:20 <karsten> what happened?! :)
14:45:29 <irl> so we had a goal of shutting down compass
14:45:31 <irl> https://atlas.torproject.org/#aggregate/cc
14:45:36 <irl> what is missing?
14:46:03 <irl> the ability to select based on more than one flag is missing, but there's an onionoo ticket for that already
14:46:26 <karsten> huh. should we raise priority of that?
14:46:36 <iwakeh> #23914
14:46:40 <karsten> and maybe include it in 1.8.0?
14:46:45 <irl> that would be good (:
14:47:06 <irl> the query that i think is missing is only selecting based on number in a single /16
14:47:17 <iwakeh> (oh, the bot doesn't produce links here.)
14:47:31 <irl> nope, "Fast exit relays (95+ Mbit/s, 5000+ KB/s, 80/443/554/1755, 2 relays per /24)"
14:47:32 <irl> that one
14:47:39 <irl> you also cannot search based on exit policy
14:47:41 <karsten> ah, that one is old.
14:47:56 <karsten> I'd say drop anything related to "fast exit".
14:48:17 <irl> ok, so then the only things we're missing is selecting based on multiple flags
14:48:32 <irl> oh, and it doesn't allow showing non-running relays currently
14:48:40 <irl> is that a useful feature?
14:48:50 <karsten> hrmm, probably not.
14:49:06 <irl> then we can turn off compass once multiple flags is implemented
14:49:30 <karsten> should we put up a notice before that?
14:49:37 <irl> probably
14:49:45 <irl> i also have not yet considered any redirects
14:49:59 <irl> i think i would prefer to just give a big notice that you can do the same things with relay search now
14:50:06 <karsten> like, announce that we're going to turn it off by december 31 and that people should try out relay search and report anything they're missing.
14:50:15 <irl> sounds great (:
14:50:24 <iwakeh> should be fine.
14:50:27 <karsten> cool!
14:50:53 <irl> the other change that was a bigger change is the new advanced search
14:50:54 <irl> https://atlas.torproject.org/#advanced
14:51:11 <karsten> do you want to submit a patch for that announcement?
14:51:14 <irl> this basically just builds up the query for you, but it exposes all the features of onionoo so that users know they exist
14:51:21 <irl> i can do a patch, yes
14:51:43 <karsten> pretty neat!
14:52:15 <karsten> much more useful than just text documentation.
14:52:29 <iwakeh> true!
14:52:31 <irl> i still need to add first_seen_days and version
14:52:37 <irl> but it's almost there
14:53:10 <karsten> sounds like a productive weekend. :D
14:53:25 <irl> heh, some of this has been cooking in my branches for a while
14:54:01 <irl> just making sure all my 2017 tickets are out of the way before the end of 2017 (:
14:54:16 <karsten> so, UX-wise, I could imagine that there are possible improvements to the three tabs "Simple Search", "Aggregated Search", and "Advanced Search".
14:54:31 <karsten> but I don't know how exactly to do it better.
14:54:48 <karsten> maybe we could ask the UX folks to take a look once all functionality is there.
14:54:56 <irl> yep, that would be good
14:55:00 <iwakeh> could be overhauled in 2018 with the support of the web-developer.
14:55:07 <irl> i also looked at internationalisation of the templates
14:55:09 <karsten> true.
14:55:14 <irl> for non-english operators
14:55:26 <karsten> that will be fun!
14:55:36 <irl> but not this year
14:56:03 <irl> the updates for relay search did get shared around twitter when nusenu spotted them
14:56:09 <irl> and i've not had any complaints of bugs
14:56:14 <irl> so i think these have gone well
14:56:22 <karsten> great!
14:56:25 <irl> https://twitter.com/satoshiPL/status/934856957462228992
14:56:25 <iwakeh> excellent
14:56:32 <irl> the data was used for an analysis ^
14:56:45 <irl> that's all i have for this topic
14:56:54 <karsten> heh, neat!
14:57:12 <karsten> okay, moving on?
14:57:38 <karsten> * Onionoo 4.4-1.8.0 release this Thursday or Wednesday? (karsten)
14:57:55 <iwakeh> depends o the tickets in 1.8.0
14:58:07 <karsten> true.
14:58:19 <karsten> shall we take a quick look?
14:58:26 <karsten> at the trac url you posted on the pad?
14:58:33 <iwakeh> yep
14:58:59 <karsten> #23348 could be postponed again. :)
14:59:11 <iwakeh> #24222 too
14:59:12 <karsten> I don't know about #24222.
14:59:20 <karsten> ok.
14:59:29 <karsten> #24384 needs to go in.
14:59:31 <iwakeh> It's the war structure improvements and would take longer to review.
14:59:37 <karsten> ok.
14:59:58 <karsten> I have been looking into #24384 for a few hours today.
15:00:06 <iwakeh> ok
15:00:09 <karsten> and I think it's not quite merge_ready yet.
15:00:18 <karsten> this + sign...
15:00:23 <iwakeh> why?
15:01:12 <karsten> I'm not certain yet, but I think jetty (and tomcat) treats + different than we do it with the patch.
15:01:27 <karsten> and if we want to keep parameters and qualified search terms in sync, we'll have to do what jetty does.
15:01:59 <iwakeh> let's discuss on ticket.  Did you comment there?
15:02:04 <karsten> I'll look more into this and... yes. not yet.
15:02:20 <karsten> let's prioritize that ticket and do it as part of 1.8.0.
15:02:35 <karsten> #16513 could wait.
15:02:44 <iwakeh> ok
15:02:47 <karsten> depends on your availability for review.
15:03:00 <karsten> but, it can wait.
15:03:01 <iwakeh> I can fit that in.
15:03:14 <karsten> or, hmm.
15:03:15 <iwakeh> depends on how much comes up with '+'
15:03:29 <irl> https://atlas.torproject.org/#search/contact:+tor
15:03:35 <irl> 55 entries
15:03:43 <irl> mostly people with + in email addresses
15:03:43 <karsten> I'm not 100% convinced that #16513 is the right thing to do.
15:03:44 <iwakeh> Maybe leave things out that are not ready yet.
15:04:21 <iwakeh> it's just tomorrow for both review and coding.
15:04:39 <karsten> agreed.
15:04:40 <iwakeh> guessing the release will happen wednesday.
15:05:12 <iwakeh> I'll add the open tickets to 1.9.0
15:05:30 <iwakeh> except 24420
15:05:33 <karsten> irl: I can't say yet what is working how and why. let's move that discussion to the ticket, okay?
15:05:58 <irl> karsten: yep, that's fine.
15:06:02 <karsten> yes, except 24384 and 24420.
15:06:40 <karsten> so, release on wednesday, deployment and announcement on thursday?
15:06:54 <iwakeh> I'm afk on Thursday.
15:06:59 <karsten> ah.
15:07:10 <iwakeh> That's why we have the meeting today :-)
15:07:14 <karsten> yes, yes. :)
15:07:25 <karsten> release tomorrow?
15:07:31 <iwakeh> what about deployment?
15:07:43 <karsten> wednesday?
15:08:03 <iwakeh> I mean regarding the complaints
15:08:21 <iwakeh> we had on the ml for the most recent release?
15:08:26 <karsten> well, I think that was resolved.
15:08:43 <karsten> wasn't the issue just a fragile parser?
15:08:44 <iwakeh> ok, deploy on the 29. is fine
15:08:56 <iwakeh> true
15:09:01 <karsten> I think it's fine to deploy on the 29th.
15:09:29 <karsten> great!
15:09:34 <karsten> moving on to the last item?
15:09:37 <iwakeh> Rather postpone exonerator
15:09:41 <karsten> * first exonerator release together with onionoo? (iwakeh)
15:09:42 <karsten> yes.
15:09:46 <iwakeh> due to the tight release schedule.
15:10:00 <iwakeh> but for sure december?
15:10:03 <karsten> do we even need a public release now?
15:10:17 <karsten> or can that wait until after the interation into metrics-web?
15:10:56 <karsten> (do we include the build_revision in exonerator's query.json?)
15:11:33 <karsten> anyway, I'd think that releasing exonerator is not a top priority at the moment.
15:12:15 <karsten> did we lose iwakeh?
15:12:41 <karsten> so, next meeting will be next thursday.
15:13:09 <karsten> is there anything else we should discuss now, in particular because of that rather long time period?
15:13:11 <irl> cool
15:13:11 <hiro> next thursday meaning in a week and a half or meaning 3 days?
15:13:18 <karsten> week and a half.
15:13:22 <hiro> ok thanks
15:13:26 <irl> oh, we need to update the wiki
15:13:32 <irl> #tor-dev -> #tor-meeting
15:13:35 <karsten> hah
15:13:41 <karsten> true. want to do that?
15:13:44 <irl> ok
15:13:49 <karsten> thanks.
15:13:57 <irl> if you have time karsten, we can discuss my remaining things for the year
15:14:10 <karsten> sure!
15:14:20 <karsten> as part of this meeting?
15:14:28 <irl> either as part of the meeting or not
15:14:33 <irl> i don't mind
15:14:39 <karsten> me neither.
15:14:47 <karsten> let's just continue.
15:15:01 <karsten> it might be relevant for others, too.
15:15:34 <irl> ok, so #23518 Turn Atlas into page on Tor Metrics, i'm happy that we've made enough progress on this ticket this year and the remaining work is just a bit of integration glue
15:15:42 <irl> would you agree with that?
15:15:56 <karsten> speaking of:
15:16:04 <karsten> did you see my suggestion to reduce header height?
15:16:16 <irl> the "purple" ticket
15:16:20 <karsten> if you agree with that, we could coordinate an update of style.css.
15:16:21 <irl> i didn't see the actual ticket
15:16:21 <karsten> yeah.
15:16:32 * iwakeh back again.
15:16:41 <karsten> irl: #24277
15:16:46 <iwakeh> still disconnected from the pad.
15:16:48 <karsten> iwakeh: welcome back!
15:17:34 <karsten> iwakeh: there, sent you backlog.
15:17:50 <iwakeh> thanks. checking ..
15:17:50 <irl> karsten: i'm not sure that i agree with your patch's implementation
15:17:54 <karsten> irl: so, I'm happy to leave things as they are, or to move forward with the integration.
15:17:59 <karsten> irl: hehe.
15:18:01 <irl> but i could update the patch to make it more sane
15:18:06 <karsten> irl: please do!
15:18:13 <irl> and then yes, we could coordinate on the deployment of that
15:18:23 <karsten> irl: I was just very careful not to mess up things.
15:18:36 <irl> heh
15:18:37 <karsten> (does your screen show 8000+px?...)
15:18:53 <karsten> anyway.
15:18:55 <iwakeh> (that's a wall)
15:19:06 <karsten> vertically!
15:19:23 <irl> maybe some retina tv on its side
15:19:34 <iwakeh> hehe
15:19:46 <karsten> so, regarding atlas/m-web integration,
15:19:49 <karsten> up to you.
15:19:53 <irl> ok cool
15:19:57 <karsten> happy to do more in 2017, happy to do more in 2018.
15:20:01 <irl> which leaves me with one ticket
15:20:16 <irl> Explain why metrics are important and what we do to make sure they're safe
15:20:19 <irl> #23169
15:20:24 <karsten> yes.
15:20:32 <irl> i've started on this, just producing a wall of text
15:20:44 <karsten> I can imagine.
15:21:05 <irl> i might split some of this between the landing page and a "read more" link
15:21:17 <irl> and i think we want to move information for potential contributors to a seperate page
15:21:33 <irl> the portal is more useful to users of the data than to those that want to contribute to producing it
15:21:44 <iwakeh> that's true
15:21:59 <karsten> do we have much for potential contributors?
15:22:06 <irl> the whole about page looks very "come join us" focussed, instead of what tor metrics provides
15:22:16 <karsten> hmm, maybe.
15:22:17 <iwakeh> we could leave that out
15:22:30 <karsten> well, we might have to leave a link somewhere.
15:22:36 <iwakeh> The contributor's topics are better on the team page.
15:22:36 <irl> I'd like to drop it to a single sentence with a link to "contribution information over here"
15:22:43 <iwakeh> yep
15:22:46 <karsten> sounds good.
15:23:08 <irl> ok cool, it sounds like we generally agree (:
15:23:19 <karsten> yes. :)
15:23:39 <karsten> so, do we need subpages for About?
15:23:45 <irl> possibly
15:23:54 <irl> i've made a ticket about exploring information architecture for the portal
15:24:04 <irl> https://trac.torproject.org/projects/tor/ticket/24422#ticket
15:24:06 <karsten> yes, I saw that, but I didn't look yet.
15:24:17 <karsten> thanks for starting that!
15:24:23 <irl> but i'm just producing copy at the moment, and then we'll work out where it goes
15:24:31 <karsten> sounds good.
15:24:42 <irl> this will probably run until the end of Q1 2018 as an ongoing thing in the background
15:25:14 <karsten> okay. just remember to release early, release often. :)
15:25:28 <irl> yep, it's going to be very iterative i think, with ux-team involvement too
15:25:37 <irl> if they have time
15:25:39 <karsten> yep.
15:25:45 <iwakeh> fine
15:25:48 <karsten> ah, speaking of!
15:26:00 <iwakeh> ux?
15:26:02 <karsten> #24260
15:26:15 <karsten> https://trac.torproject.org/projects/tor/ticket/24260
15:26:30 <karsten> we got feedback from ux folks.
15:26:41 <iwakeh> ok.
15:26:51 <karsten> I'd like to do something with that by the end of the month, too.
15:27:06 <karsten> if you agree or disagree with something, please comment on the ticket.
15:27:21 <irl> the mockup looks good
15:27:55 <irl> ok well, that's all the things i plan to do this year
15:28:03 <karsten> agreed. I'm not sure why I didn't use all the available space earlier.
15:28:28 <karsten> irl: sounds great!
15:28:28 <irl> i've put it a lot of time last week, i probably won't have so much time until next year, except for the information architecture start and writing copy for the about goal
15:28:56 <karsten> be sure to include a little bit of time when onionoo updates come in.
15:28:56 <irl> oh, i did file one other ticket for ux-team
15:29:16 <irl> https://trac.torproject.org/projects/tor/ticket/24399#ticket
15:29:58 <irl> yes, i'll have time for onionoo updates, but i think i have already handled most of it, like making $ signs optional in the family
15:30:16 <karsten> ok.
15:30:23 <iwakeh> that's a good idea.
15:30:31 <hiro> irl that would actually be great also for other projects
15:30:39 <irl> that was my thinking
15:30:41 <iwakeh> +1
15:30:51 <hiro> we are also doing the styleguide
15:30:57 <hiro> we could include a set of icons there
15:31:02 <irl> i was hoping this could be included in the styleguide (:
15:31:32 <hiro> antonela is mocking that we should have that online soon.
15:31:32 <irl> that would be perfect, so other projects would know it exists
15:31:37 <irl> awesome
15:31:41 <hiro> yes!
15:32:07 <karsten> speaking of re-using things in other projects,
15:32:15 <karsten> I somewhere read the suggestion to merge glossaries.
15:32:22 <karsten> which makes a lot of sense, too.
15:32:31 <iwakeh> definitely
15:32:33 <irl> yes, at first our glossary was not included, but i added it to the ticket when i spotted it
15:32:52 <irl> i was also thinking of using the glossary to produce the atlas tooltips for things
15:33:01 <iwakeh> have writers maintain glossaries.
15:33:10 <iwakeh> or one.
15:33:11 <irl> no idea how to do that technically though, unless there's a JSON glossary service
15:33:22 <karsten> which could exist.
15:33:40 <karsten> anyway, I think I gotta run now.
15:33:44 <irl> if there was, metrics-bot could consume it to make it available in #tor for support queries
15:33:51 <karsten> iwakeh: thanks for updating the agenda pad with exonerator things.
15:33:51 <irl> ok cool
15:33:59 <iwakeh> ok, time release by mail.
15:33:59 <karsten> irl: yes, good idea.
15:34:14 <karsten> exonerator release or onionoo release?
15:34:20 <iwakeh> onionoo.
15:34:24 <karsten> ok.
15:34:39 <karsten> thanks, everyone. talk more in ~1.5 weeks! bye! :)
15:34:43 <Sebastian> hey
15:34:43 <irl> bye!
15:34:45 <Sebastian> quick question
15:34:48 <karsten> hi Sebastian!
15:34:50 <iwakeh> thanks! bye,bye
15:34:52 <hiro> bye
15:34:54 <iwakeh> hi!
15:35:03 <iwakeh> webstats is in the making :-)
15:35:05 <Sebastian> when can we turn off the weblogs stuff on wendelboi?
15:35:13 <Sebastian> ah :)
15:35:15 <karsten> not there yet.
15:35:20 <Sebastian> ok, cool
15:35:25 <karsten> sorry. :)
15:35:32 <Sebastian> np
15:35:33 <iwakeh> specifiction.
15:35:37 <karsten> gotta run! bye!
15:35:40 <Sebastian> bb
15:35:41 <karsten> #endmeeting