16:59:34 <hellais> #startmeeting
16:59:34 <MeetBot> Meeting started Mon Feb  1 16:59:34 2016 UTC.  The chair is hellais. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:59:34 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
16:59:41 <hellais> here we go, who is here?
16:59:56 <landers> here
17:00:01 <willscott> hi
17:00:17 <sbs> hi!
17:01:23 <anadahz> hi
17:01:50 <hellais> excellent, so what have you been up to last week?
17:03:47 <poly> hey all
17:03:50 <landers> massaged the pipeline, now adding some things to the incoming->reports-raw/yaml script
17:03:51 <landers> eof
17:04:52 <hellais> I have been working on the hopefully final iteration of the pipeline that integrates the normalisation hodgepodge does in his ooni-lyzer, writes to one report per report_id. Then I reviewed the specifications in ooni-spec to include the normalisation done in the pipeline and started implementing the web_consistency test helper and test in the probe as well as added support for JSON in ooni-probe and ooni-bac
17:04:59 <hellais> kend. EOF
17:05:36 <hellais> s/web_consistency/web_connectivity/
17:05:49 <hodgepodge> Great! That's good news. Thanks for integrating both pipelines into one!
17:08:30 <simonv3> I spent some time last week fiddling with usability and just general bugs on the frontend-api.
17:09:36 <simonv3> EOF
17:10:20 <sbs> I cleaned up OONI implementation of MeasurementKit. Hellais, Nuke, and I had a conference call with developers of Portolan to discuss a roadmap to integrate MeasurementKit into their Android App. EOF
17:11:59 <simonv3> I’m not sure what the deal is with this week - bnvk_ added quite a lot of stuff to work on, though I’m not sure if he’ll be tackling it or whether I should help out with it
17:12:44 <anadahz> I was in travelling mode, previous weeks i  worked setting up the hardware infrastructure, setting some bureaucratic/firewall policies on the university servers. Hopefully we can move away from AWS.
17:12:47 <anadahz> EOF
17:13:29 <hellais> simonv3: I think he has plans on tackling them since they are mostly design related, but you should probably speak to him about that.
17:15:37 <simonv3> Been having a bit of a hard time getting a hold of him :/ Think it’s fosdem so he’s traveling?
17:16:03 <hellais> simonv3: yeah he went to fosdem this weekend, but he should have gotten back today.
17:16:15 <simonv3> Cool, I’ll wait for word from him
17:18:41 <hellais> so are there some things people would be interested in discussing?
17:21:04 <MightyOctopus> [13measurement-kit] 15DavideAllavena 04force-pushed 06issue-297 from 14898df81 to 14f90a398: 02https://git.io/vgfHI
17:21:04 <MightyOctopus> 13measurement-kit/06issue-297 14f90a398 15Davide Allavena: default parameter in the constructor.
17:21:09 <MightyOctopus> [13measurement-kit] 15DavideAllavena opened pull request #349: default parameter in the constructor. (06master...06issue-297) 02https://git.io/vgfHL
17:21:32 <hodgepodge> I think that nullnode has a question for all of us regarding whether or not ooni-probe tests can serve as a warrant canary
17:22:47 <landers> hodgepodge: where?
17:23:47 <hodgepodge> He joined #ooni a little while ago in order to ask a question.
17:26:03 <landers> nullnode: come back with a warrant question
17:26:39 * hodgepodge finger guns
17:26:46 <hodgepodge> Anyway, I think he's gone. I'll direct him to the mailing list.
17:28:49 <anadahz> hodgepodge: weird it doesn't apear to my backlog
17:31:46 <anadahz> how far is the OONI API?
17:33:35 <hellais> anadahz: I would say that we have reached a point where the MVP is implemented. I think I am going to merge tomorrow the pending branches and we can continue the final polishing on new feature branches. With brennan some days ago we made a list of all the things that need to happen before we publish it.
17:33:58 <anadahz> hellais: awesome!
17:34:19 <anadahz> is this the version in: http://api.ooni.io/world/ ?
17:34:45 <hellais> I am also thinking that it may make more sense to rename it to the OONI Explorer and have it reside under explorer.ooni.io, since the API has taken a direction of being very specific to the visualisations that we implement in this frontend and not necessarily something that much consumable for other applications
17:35:12 <hellais> anadahz: yes, what is at that address is the latest version
17:38:43 <hellais> I also found while looking through the results something interesting, that I am not sure exactly how to read, but it's worth sharing I think
17:38:46 <simonv3> Agreed with renaming it
17:38:53 <sbs> hellais: what is the difference between /world and /explorer
17:38:55 <sbs> ?
17:38:58 <MightyOctopus> [13measurement-kit] 15DavideAllavena created 06issue-348 (+1 new commit): 02https://git.io/vgfda
17:38:58 <MightyOctopus> 13measurement-kit/06issue-348 1434cd4c7 15Davide Allavena: traceroute error namespace moved to 4000.
17:39:15 <hellais> basically Lao, Indonesia and New Zealand have a very similar fingerprint for the responses of the HTTP Invalid Request line test: http://api.ooni.io/measurement/WBixAGaXiATvvWLpW1WeNoBHI6h8T8SmWQmwHS1jEoKgI7RfRUcSQ4Gso5ZH8b8J
17:39:21 <hellais> http://api.ooni.io/measurement/2015-05-04qliggmamplzhdxvxqynquwfxztsvunwcegsytdvs
17:39:43 <simonv3> Sbs explorer is just the list of all the measurements for filtering
17:39:57 <hellais> http://api.ooni.io/measurement/2015-09-24qaefcicxkzbsddiryyqawwtotxolzdvgkddsrbwb
17:40:09 <simonv3> Doing your own searching on and such.
17:40:16 <hellais> which seems to indicate that there may be similarity in the type of gear used there
17:40:51 <simonv3> World is a bit more exploitation oriented if you don't really know what you're looking for.
17:41:39 <simonv3> Hellias: is the idea that the api available under /explore/ is published? Should we link to it?
17:41:50 <sbs> simonv3: I see, thanks!
17:42:11 <simonv3> And maybe we should rename that route? (Because it's too similar to explorer)
17:42:57 <sbs> hellais: perhaps stupid question, but: why the first URL you shared is only a - I think - hash and the others are a date followed by possibly again an hash?
17:43:49 <hellais> sbs: because of inconsistencies in the report_id format across different versions of ooni-probe.
17:44:29 <sbs> hellais: ah, ok, thanks
17:44:50 <sbs> simonv3: yes, maybe
17:45:42 <hellais> simonv3: Yeah I think we could link to the /explorer (the strongloop explorer), but I would not link it from the web interface of the explorer since this frontend is not oriented towards developers, but it should probably be in the developer documentation
17:46:40 <simonv3> Cool
17:46:48 <sbs> ah, cool that there are both /explore/ and /explorer/ :-)
17:47:15 <hellais> it is a bit confusing though
17:47:16 <sbs> BTW, I really like the way in which results are presented, for example here http://api.ooni.io/measurement/Opq7y1GRbXxbSL7NiGh1ch78IYnqRjfCdvkS2UFDYpawe6mT1EN44mMmnqPxfQuQ?input=http:%2F%2Fanonym.to
17:48:08 <sbs> also, in the latter URL I shared, why I get nothing if I remove the final query string?
17:48:46 <sbs> (I was thinking that the query was a way to take just a subset of the measurement identified by the hash)
17:49:04 <sbs> hellais: yes, it is!
17:49:09 <hellais> sbs: because measurements are currently identified by a composite primary key of the report_id and the input
17:49:21 <sbs> ah, no, sorry: it actually replied after long time
17:51:17 <hellais> ok so I think we should move into the next steps
17:51:56 <anadahz> Is the the Highlights page (http://api.ooni.io/overview/) manually added or auto-generated?
17:52:29 <hellais> I will continue working on the web_connectivity test, follow the development of the ooni-api and merge the pending feature branches into master and work on cleaning up all the dead code from the ooni-pipeline and integrating that into master as well
17:52:54 <hellais> anadahz: manually generated and it's currently incomplete.
17:53:16 <hellais> anadahz: see this ticket for details on what it's going to look like: https://github.com/TheTorProject/ooni-api/issues/28
17:54:49 <anadahz> hellais: so it's like the blog posts?
17:54:54 <MightyOctopus> [13measurement-kit] 15DavideAllavena pushed 1 new commit to 06issue-297: 02https://git.io/vgfAk
17:54:54 <MightyOctopus> 13measurement-kit/06issue-297 1460cfc2b 15Davide Allavena: Merge branch 'master' into issue-297
17:55:23 <anadahz> since it's manually generated?
17:56:01 <hellais> anadahz: in a way I guess. It's meant to provide a bit of context around the most interesting measurements and have a place for people to start from to find interesting results.
17:56:23 <anadahz> but will display only the
17:56:46 <anadahz> section mentioned in: https://github.com/TheTorProject/ooni-api/issues/28 ?
17:57:28 <anadahz> is this part of the requirement? Otherwise we could merge it with the blog posts
17:57:31 <simonv3> Hellias you could also include you're observation about those three measurements you mentioned earlier, encourage people to explore like that
17:57:53 <hellais> anadahz: that is what we have come up with currently. The purpose is to not drown the user with a bunch of information, but be synthetic
17:58:09 <hellais> anadahz: I don't see what you mean with "merge it with the blog posts"
17:58:48 <hellais> simonv3: yeah that was the idea
17:59:31 <travis-ci> TheTorProject/ooni-probe#1366 (feature/web_connectivity - 2e18fd1 : Arturo Filastò): The build failed.
17:59:31 <travis-ci> Change view : https://github.com/TheTorProject/ooni-probe/compare/15ee77cc3f8e...2e18fd1d5d0f
17:59:31 <travis-ci> Build details : https://travis-ci.org/TheTorProject/ooni-probe/builds/106262232
18:02:18 <sbs> Regarding my next steps, I'll probably release MeasurementKit 0.1.2 (https://github.com/measurement-kit/measurement-kit/milestones/release-0.1.2), and then move on to open a pull request to merge the code that integrates Tor inside MK
18:02:22 <sbs> EOF
18:06:53 <anadahz> Will continue on lepidopter aiming for a release this week. Update OONI website and continue adding documentation/testing ooniprobe tests.
18:07:13 <MightyOctopus> [13measurement-kit] 15bassosimone opened pull request #350: Traceroute error namespace moved to 4000. (06master...06issue-348) 02https://git.io/vgfpH
18:07:41 <hellais> I have to run out, so I am going to leave the meeting pending and end it when I get back online.
18:07:43 <anadahz> Need to to catch up since I was in travelling mode.
18:07:46 <anadahz> EOF
18:08:02 <landers> will continue on pipeline, ops scripting EOF
18:14:31 <bnvk_> halloo, are you still in the meeting?
18:15:09 <bnvk> this time slot is the absolute worst for me given my normal daily flow
18:15:18 <bnvk> I totally meant to be here this eve on time
18:16:00 <anadahz> bnvk: Hi!
18:16:15 <travis-ci> TheTorProject/ooni-probe#1367 (feature/json_reports - 413d6b9 : Arturo Filastò): The build failed.
18:16:15 <travis-ci> Change view : https://github.com/TheTorProject/ooni-probe/compare/35e878c6e3c7...413d6b97eb0e
18:16:15 <travis-ci> Build details : https://travis-ci.org/TheTorProject/ooni-probe/builds/106262285
18:16:34 <bnvk> anadahz: heya
18:16:52 <bnvk> anywho, I'll say my update if anyone is around
18:18:13 <bnvk> I had a great working session with Artuto last week on Friday
18:18:28 <bnvk> we speced out a bunch of front-end polishing and refinement to do
18:18:56 <bnvk> was great digging into the stuff simonv3 and everyone else has done- really great stuff :)
18:19:00 <bnvk> simonv3: you around?
18:21:01 <bnvk> anywho, I plan to make some progress on those issues this week, as well as coordinate what makes sense for simonv3 to continue on
18:21:12 <bnvk> EOF
18:28:22 <anadahz> bnvk: there was some discussion regarding the API if you don't have backlog I can sent the relevant sections
18:29:28 <nullnode> hodgepodge: ty for the intro! I'll email the list
18:36:01 <bnvk> anadahz: that would be great
19:05:31 <hellais> #endmeeting