15:01:54 <karsten> #startmeeting metrics team
15:01:54 <MeetBot> Meeting started Tue Nov 12 15:01:54 2019 UTC.  The chair is karsten. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:01:54 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
15:02:10 <karsten> https://storm.torproject.org/shared/5h1Goax5eNusxjXJ_Ty5Wl7hFR1uqCReUiN8xdlBG8T <- agenda pad
15:02:14 <karsten> speaking of,
15:02:29 <karsten> should we move this pad to nextcloud?
15:02:39 <gaba> yes
15:02:40 <irl> i have no idea what nextcloud is or does
15:02:43 <irl> or how to log in to it
15:02:50 <karsten> it's our new storm!
15:02:52 <gaba> irl: i sent you an invite
15:02:59 <irl> this seems to have happened without me noticing
15:03:06 * gaba is going to resend it
15:03:12 <karsten> this is not urgent, AFAIK.
15:03:16 <irl> i thought we were still deciding what nextcloud was for, and if we even wanted it
15:03:27 <irl> but ok i guess i have to catch up
15:03:30 * karsten leaves this part to gaba.
15:03:52 <gaba> irl: we had a meeting in stockholm and then we decided to go with nc.torproject.net
15:04:20 <gaba> irl: i just resent you the mail with the invite
15:04:22 <karsten> my understanding is that it's already decided that storm will be shut down in 3-4 months.
15:04:28 <gaba> yes
15:04:38 <gaba> the same with nc.riseup.net that we were using for testing
15:04:39 <irl> ok cool
15:05:00 <karsten> so, should I create this new pad? and how would I do that?
15:05:10 <gaba> yes please
15:05:11 <karsten> or would you create that, gaba, and invite us?
15:05:14 <gaba> mmm
15:05:19 <gaba> I can create it
15:05:29 <irl> the pad should be public
15:05:33 <gaba> I will copy stuff from the pad now and then send you all the link
15:05:34 <gaba> yes
15:05:37 <karsten> right.
15:05:38 <gaba> It will be public
15:05:58 <karsten> by invite us I meant tell us how to find it.
15:06:03 <gaba> yes
15:06:12 <karsten> okay, cool!
15:06:45 <irl> Enhanced security is enforced for your account.
15:06:48 <irl> i guess this isn't happening today
15:07:11 <karsten> I found FreeOTP really useful.
15:07:50 <karsten> anyway. we don't have to rush this move.
15:08:19 <karsten> maybe it works for the next meeting, but if not, for the one after that.
15:08:49 <karsten> okay, shall we move on?
15:08:53 <irl> ok
15:09:05 <karsten> Onionoo upgrade to 7.0-1.21.1 (karsten)
15:09:17 <karsten> did you see my message about hacking the new backend host? ;)
15:09:24 <irl> i did
15:09:35 <irl> the workaround for the problem is to add to your .ssh/config https://paste.debian.net/plain/1115903
15:09:39 <karsten> do you have time, maybe after this meeting, to upgrade properly?
15:09:48 <irl> yes
15:10:10 <irl> (i did think for a while, the workaround was the fix, but the fix should be part of metrics-cloud not part of the environment it's run in)
15:10:41 <karsten> okay. happy to try that.
15:11:31 <irl> the hack seemed to be an efficient hack though
15:11:35 <karsten> any other topics before we go to the roadmap?
15:11:36 <karsten> hehe
15:11:37 <irl> and it'll be undone by the ansible next run
15:11:46 <karsten> yes, that was my understanding!
15:12:00 <irl> actually yes, another topic
15:12:19 <karsten> Exit scanner (irl)
15:12:29 <irl> i rewrote the exit scanner in python
15:12:44 <irl> just the scanner portion, but it's done
15:12:52 <karsten> hey hey
15:13:06 <karsten> what does it do?
15:13:14 <karsten> what data does it use for input, what does it output?
15:13:21 <irl> we were starting to look like we were running out of time so i started looking around at easier ways to attack this problem
15:13:57 <irl> it spawns its own tor process and uses the consensus data from that tor process to filter for all exit relays we could test with active measurements
15:14:22 <gaba> irl: where is the code for it?
15:14:30 <irl> then it builds circuits to each of those exit relays, and when the circuit is ready (using an event driven architecture) it makes a request for check.torproject.org
15:14:40 <irl> then it logs the ip address and the fingerprint to an output file
15:14:53 <irl> the missing part is something that merges the log file with the previous exit list
15:15:08 <irl> dropping old measurements as they age out of the list, and adding new ones
15:15:38 <irl> the code is a modified version of https://github.com/NullHypothesis/exitmap/blob/master/src/modules/checktest.py
15:16:00 <karsten> is it ready for review?
15:16:01 <irl> the modification is to log whether or not check already knows about it
15:16:13 <irl> it's not ready for review, i finished it just before the meeting
15:17:04 <karsten> sounds pretty cool!
15:17:36 <irl> we would still need to modify check.tpo to use the http url as its source for the exit list
15:17:46 <irl> and we still need to write a dns server
15:17:55 <irl> but this is a good boost
15:18:35 <karsten> yes, this is an important step.
15:18:52 <karsten> when it's ready for review, please let me know. I'd like to take a look and give feedback.
15:19:18 <irl> yes, i will ask you to review once i've written the merger script
15:19:24 <irl> should be tomorrow afternoon
15:19:27 <irl> or evening
15:19:28 <karsten> awesome!
15:20:23 <irl> ok, on to the roadmap?
15:20:32 <karsten> fine by me!
15:21:04 <gaba> ok
15:21:10 <karsten> Roadmap - how are we doing? (gaba)
15:21:31 <karsten> fortunately, I was able to move some cards to done this weekend.
15:21:37 <irl> ok, so we drop #32262 and go straight to having #32264 being in progress, which now is actually talking about modifications to exitmap to do the same thing (but way easier)
15:22:54 <irl> #32263 is no longer required
15:23:12 <irl> #32265 is now the merger of exitmap results with the previous exit list
15:23:16 <karsten> should there be a column for cards we're not doing anymore?
15:23:32 <karsten> or is that the done column?
15:23:46 <irl> i have no preference
15:23:55 <gaba> it is the icebox
15:24:04 <gaba> it goes back to icebox or we close it if we will not ever do it
15:24:20 <karsten> well, more like trash.
15:24:31 <karsten> we're not keeping trash in the ice box, are we?
15:24:32 <gaba> #32263 should be close?
15:24:47 <irl> yeah #32263 wontfix
15:24:55 <gaba> in that case is closed unfixed/not-implemented
15:25:03 <irl> also #32262 is wontfix
15:25:04 <gaba> ok. We close the ticket and we move the card to done
15:25:05 <gaba> :)
15:25:10 <karsten> done is good.
15:26:35 <irl> so i have #32264 and #32265 in progress
15:26:47 <karsten> should we also have a card for comparing exit lists and the new scanner output?
15:27:19 <karsten> they're not going to be a perfect match, but does the new scanner produce comparable results?
15:27:36 <irl> doing set stuff with the addresses?
15:27:41 <irl> i think yes, we should do that
15:28:28 <karsten> want to create a ticket for that? I can do the comparison, if needed.
15:29:38 <karsten> guess we can talk about that next week. this is step 2 before step 1 is done.
15:29:38 <irl> #32473
15:29:49 <karsten> great!
15:30:40 <karsten> alright. I'm out of cards in progress at the moment.
15:31:02 <karsten> which is good, because I finally got that index.json stuff out of the way.
15:31:10 <irl> you could already start on a system for comparing exit lists
15:31:29 <karsten> they're the same format?
15:32:12 <irl> yeah
15:32:26 <irl> drop-in replacement
15:32:29 <karsten> sure, I can do that.
15:32:41 <karsten> I'll grab #32473 then?
15:33:22 <irl> sounds good
15:33:50 <karsten> done (on the board).
15:34:03 <karsten> oh, #31071.
15:34:16 <karsten> this was a bit sad when I tried to deploy this on the server.
15:34:29 <karsten> query times went up from 2.5-3 seconds to up to one minute...
15:34:38 <irl> oh dear
15:34:53 <karsten> but debugging this might take some time.
15:35:19 <karsten> I guess it's something for the icebox. it would block other things if it stays in the backlog.
15:35:30 <irl> it's possible that we solve this in the future through exit scanner redundancy
15:35:33 <karsten> could be an easy fix, could be database hacking.
15:35:45 <karsten> oh!
15:35:53 <karsten> that would be pretty neat.
15:36:09 <karsten> putting it on ice.
15:36:12 <irl> cool
15:36:59 <karsten> okay. I might need to take a small break tomorrow and thursday, so I think I'm good with one active card.
15:37:22 <irl> no problem
15:37:44 <karsten> anything else on the roadmap?
15:37:57 <irl> not for me
15:38:06 <gaba> no
15:38:11 <karsten> okay.
15:38:21 <karsten> then let's end this meeting and head over to onionoo deployment, okay?
15:38:57 <irl> ok
15:39:01 <gaba> ok
15:39:09 <karsten> next meeting next thursday?
15:39:20 <gaba> November 21st?
15:39:23 <karsten> yes.
15:39:46 <gaba> ok
15:40:24 <irl> ok
15:40:27 <karsten> great!
15:40:42 <karsten> talk to you next week then! bye! o/
15:40:45 <karsten> #endmeeting