16:59:45 <hellais> #startmeeting
16:59:45 <MeetBot> Meeting started Mon Jun 29 16:59:45 2015 UTC.  The chair is hellais. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:59:45 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
16:59:49 <aagbsn> I'm here
16:59:54 <hellais> excellent!
16:59:56 <hellais> who else is here?
17:00:59 <anadahz> hi
17:01:16 <anadahz> #help
17:01:25 <sbs> anadahz: hi!
17:01:36 <hellais> sbs: funky charset
17:02:05 <sbs> Yes, from mobile for a few more minutes;)
17:02:35 <hellais> 🙉
17:02:55 <anadahz> hellais: ?
17:03:10 <anadahz> let's start
17:03:17 <hellais> sounds good
17:03:22 <hellais> so what went on last week?
17:03:25 <hellais> who wants to start?
17:03:52 <aagbsn> I can give a brief tl;dr
17:04:11 <aagbsn> basically I reviewed the current draft of the OTF OONI propopsal and added some feedback
17:04:30 <aagbsn> I think we could link to it here or on the list shortly and ask for some feedback soon - thoughts?
17:04:58 <aagbsn> the rest of the things I did this week were not OONI related
17:05:16 <aagbsn> end of tl;dr
17:05:37 <hellais> aagbsn: cool thanks.
17:06:32 <hellais> if people are interested in seeing the proposal they should send me an email, but I don't think it's ideal to make public the current version of the proposal since it contains confidential information.
17:07:54 <aagbsn> oh, I wasn't aware
17:08:10 <aagbsn> I mean, it's shared on google :P
17:08:37 <aagbsn> who's next?
17:10:34 <hellais> since sbs is on mobile I guess I shall go
17:10:51 <hellais> so anadahz have been to the citizenlab summer institute during this week
17:11:08 <hellais> it was a very fruitful and interesting event where a lot of interesting discussions and encounters were made
17:11:29 <hellais> I will give a couple highlights and perhaps defer further discussion to the later part of the gathering
17:11:59 <hellais> we identified a series of potential probe operators from a variety of different countries
17:12:36 <hellais> when we get back they will receive information on how to start gathering measurement data with lepidopter
17:12:54 <aagbsn> awesome
17:13:00 <aagbsn> great news
17:13:10 <aagbsn> any particular regions you might highlight?
17:13:53 <hellais> we also had a good session on the citizenlab test-lists and defined a roadmap for development that I will soon convert into tickets on the issue tracker (as soon as I get collaborator status and my tablet resuscitates)
17:14:38 <hellais> aagbsn: some people from east/west and south africa have expressed great interest
17:15:04 <hellais> I think we will start with south africa since it's the lowest risk one and the person there is very interested
17:15:16 <hellais> but there are plenty more also
17:17:11 <anadahz> Our article about the Greek censorship is going to be published at USENIX FOCI workshop (https://www.usenix.org/conference/foci15/workshop-program). A blog version of the article can be found here: https://ooni.torproject.org/post/eeep-greek-censorship
17:18:01 <sbs> anadahz: great!
17:20:09 <anadahz> During citizenlab summer instritute we have worked with  willscott: to add some zmap and sattelite datasets into OONI pipeline/API
17:20:36 <aagbsn> satellite?
17:21:01 <aagbsn> (is that a tool or do you mean space related datasets?)
17:21:23 <anadahz> aagbsn: http://satellite.cs.washington.edu/
17:22:41 <aagbsn> aha
17:23:24 <anadahz> Satellite curently includes a DNS resolution data set perhaps willscott (if he's around) can provide some more info about the project
17:24:18 <anadahz> I had some really nice feedback about lepidopter and RasPis it seems that we are on a good track
17:24:40 <anadahz> I was thinking what would be a nice way to monitor all RasPis managed by OONI
17:24:56 <anadahz> hellais: proposed logstash
17:26:25 <willscott> hi
17:26:34 <hellais> anadahz: with these commands you setup logstash on the box and send your logs to the oonipipeline-ng logger: https://github.com/TheTorProject/ooni-pipeline-ng/blob/master/tasks.py#L87
17:26:47 <hellais> if you want I can create a new dedicated endpoint just for the lepidopter stuff
17:27:08 <hellais> it sort of sucks that it's a cloud service I guess, but it's extremely convenient to setup and use
17:27:11 <hellais> willscott: hey!
17:27:20 <willscott> satellite in one sentence is trying to figure out how much of a view we can collect regularly from one vantage point. we zmap the ipv4 space to find open dns resolvers, and then resolve alexa top 10k domains against them
17:27:40 <willscott> it looks like in the ooni json format it'll be about 120gb / week of reports
17:28:43 <hellais> anadahz: sorry I got confused. I was thinking papertrail and not logstash. Papertrail is much simpler to setup than logstash
17:28:59 <anadahz> I would be happy to evaluate/consider other lightweight monitoring tools as well..
17:29:18 <anadahz> willscott: sorry keep on..
17:29:52 <willscott> i think that's about it :) happy to answer questions though
17:30:09 <hellais> willscott: I think we may be able to do a bit better off if we remove a bit of the redundancy from the json, though 120 GB/week is not really that much
17:31:02 <willscott> hellais: agreed. it probably compresses well in transit too, so it's only if you care about resting size
17:31:07 <hellais> willscott: I would say for the time being we proceed by having you upload one batch of the reports and look at analysing that
17:31:35 <willscott> i'll do that later today
17:31:38 <hellais> once we are extracting useful information from that first batch we will then setup the automatic upload of 120 GB/week reports
17:32:35 <hellais> willscott: excellent!
17:36:46 <sbs> During last week, nuke and I worked together to merge a pull request on measurement-kit-app-ios (https://github.com/measurement-kit/measurement-kit-app-ios/pull/14)
17:37:00 <sbs> I also contributed a bit to the OONI proposal
17:37:15 <sbs> that's it
17:38:33 <hellais> great thanks
17:38:46 <hellais> here are some numbers for your pleasure:
17:38:46 <hellais> SELECT count(*) FROM (
17:38:46 <hellais> SELECT DISTINCT "probe_cc"
17:38:47 <hellais> FROM "headers") AS counts
17:38:51 <hellais> 75
17:42:10 <hellais> so is there something in particular some of you wanted to talk about?
17:43:55 <sbs> hellais: not specifically, I'd just add that I'm working on the proposal this evening to help refine it
17:46:19 <hellais> ok good
17:46:26 <hellais> we also plan to get more work done on that
17:46:45 <hellais> as well as see if we can also talk a bit to the citizenlab people
17:53:01 <hellais> so next steps?
17:53:26 <hellais> I will be working on following up on all the conversations had in canada when I get back to Italy the day after tomorrow
17:53:39 <hellais> moreover I shall outline the roadmap for the test-lists project
17:53:51 <hellais> finish the proposal and submit it to OTF (this would actually be priority 1)
17:54:14 <hellais> and move the organisation of the hackathon forward
17:54:48 <hellais> (I am meeting on the 4th or 5th with the designers of the announcement page to finish that and start making it into a website)
17:54:50 <hellais> </end>
17:55:53 <anadahz> 'll work with hellais on the proposal
17:56:42 <anadahz> release the beta lepidopter image: ETA end of week (when I'm back)
17:59:15 <hellais> great thanks
18:00:09 <anadahz> We are currently build an OONI cluster.. if anyone has any spare servers to give away will be really nice!
18:02:26 <hellais> anybody else want to share somthing?
18:06:35 <hellais> well.. in that case
18:06:40 <hellais> thanks for attending!
18:06:45 <hellais> see you next week
18:06:46 <hellais> #endmeeting