17:01:07 #startmeeting 17:01:07 Meeting started Mon Sep 14 17:01:07 2015 UTC. The chair is hellais_. Information about MeetBot at http://wiki.debian.org/MeetBot. 17:01:07 Useful Commands: #action #agreed #help #info #idea #link #topic. 17:01:36 so 17:01:39 let's start this thing 17:01:41 who is here? 17:01:50 * sbs is here 17:01:59 hello 17:02:02 * anadahz here 17:04:18 brennan and joe are also here in the flesh 17:04:20 hallo 17:04:23 there we go 17:04:30 so let's begin with some report backs? 17:04:36 who wants to go? 17:05:48 ok, i'll start 17:05:51 i looked examples and lantern test, tried psiphon test copying lantern, modified process.py, created a more generic socksproxy test, created other psiphon test based on the modified process.py and socksproxy test. 17:05:58 EOF 17:06:07 best report back ever 17:06:35 I will state my business then 17:07:12 I mainly did a lot of fluff around organising the ADINA hackathon including reaching out to various journalists to get them to write a piece or get us some media presence 17:07:24 Started figuring out the accomodation and travel stuff 17:07:50 Reached out to people that should be coming to it and reminded them of the event and helped them coordinate their coming there 17:08:13 I also had a bunch of meeting with various people we are either working with now or may work with in the future 17:09:00 on the dev front I started implementing the Tor test #17045, but didn't do much progress on that 17:09:57 I also discussed the UX work to be done by brennan with him, but he will tell you more when he joins in some minutes 17:10:00 EOF 17:10:10 (he is reading from my screen and is fixing his irssi) 17:10:23 hello 17:10:30 quick report back: 17:11:02 sbs merged in the final branch for all the argument parsing features, working on bugfixes now 17:11:08 EOF 17:11:13 working on the monitoring with munin + ansible. got it on the roadmap as will-be-done by next friday. 17:11:16 EOF 17:12:03 bnvk: excellent! 17:12:12 Worked on improving MeasurementKit (thanks to hellais and andahz for the review), merged network-meter as poly said, some progress on finalizing the adina15 site 17:12:13 aloha 17:12:26 EOF 17:12:52 ok so my turn 17:13:44 This week I have worked in bringing up the 0.1 release of the lepidopter RasPi image 17:14:32 I have finally made a final implementation that can reproduce functional images 17:15:05 given the fact that the underlying hardware support any arm* architecture 17:15:48 hodgepodge: hey, we were just in the weekly gathering at the report back stage. I you have some pipeline report backing to be done feel free to do so 17:16:01 (vasilis though is still typing so wait for his EOF) 17:16:15 (vasilis == anadahz) 17:16:36 The lepidopter implementation change to a type of "modular" design: adding a FHS to the image. 17:17:40 Apart from this I tested the release in almost all RasPi platforms and all that have network and were sold to the market. 17:17:47 anadahz: FHS == F? Hidden Service? 17:18:18 hellais: Filesystem Hierarchy Standard 17:19:00 anadahz: why do we need that? 17:19:34 hellais: there are numerous of scripts, cronjobs, and files that are required 17:19:37 ah I see what you mean 17:19:42 and this is the most clean way 17:19:49 in the sense that you can take a directory structure with some files and they will be put on the image? 17:19:57 hellais: exactly 17:20:04 anadahz: gotcha 17:20:57 I have worked on the help section of the side providing installation instructions for Linux, Mac*, Win* OSes 17:21:27 However I'm still lacking a place on where to host this image. 17:22:51 Apart from the lepidopter things I have reached out to various groups and people for the ADINA event and reviewed some PRs 17:23:19 there are more things that I forget but we should consider this EOF atm. 17:23:34 anadahz: when you have it ready and it's on a place where I can scp it from I will put on a torproject server 17:23:49 hellais: it's ready 17:24:15 anadahz: epic! Let's discuss this out of band 17:25:11 i'll go next. my name is Brennan. I'm a designer / UX person who does a bit of code things as well 17:26:02 i'm being brought on to help spec out a REST API and a web based data visualization interface for OONI data 17:26:49 we're currently calling this "OONI Explorer" and will be making design interfaces and things that look pretty to eyes and make sense to less technical peoples brains over the next couple months 17:27:16 nice to meet all you. EOF 17:27:39 most excellent 17:27:47 is there somebody still wanting to report back? 17:29:02 I guess not 17:29:05 Sure. My name is Tyler, I'm currently working a little bit on security-related stuff, and will be helping make some of the metrics aggregated by OONI more accessible to developers, and non-technical personnel over the course of the next few weeks. 17:29:08 oh nice 17:29:13 Awh yeah. 17:29:41 Over the course of the past week, I've been reviewing the OONI architecture, specifically facets surrounding ooni-pipeline, and ooni-pipeline-ng. 17:30:22 I'm currently waiting for additional information on how to connect to the backend services used by oonib, specifically how to work with the EMR cluster used to store reports. 17:31:23 If someone could point me in the right direction with regards to how current iterations of the pipeline work at a high-level, it would be greatly appreciated. I've set-up an OONI probe in the U.S. and am in the process of setting up an instance of the ooni-pipeline on a VPS to help myself get more familiar with the codebase. 17:32:04 I'll be presenting my findings with a team at Hack The North in Waterloo, Ontario, Canada with @hugg, and hopefully I'll be able to help out with ADINA15 from here. 17:32:41 If that is an option, I'd be working on data API improvements, which is strongly related to what I'll be working on at HtN. 17:33:43 Apart from that, I have no additional ooni-related updates at this time. If it's useful, the instance of ooni-probe I am running is going to be in an environment running an i2p relay. 17:33:46 EOF. 17:34:05 good 17:34:38 I have some feedback on that, but first does anybody else have some things they would like to talk about or think we should discuss together? 17:38:13 well in that case I wanted to comment on a couple of things hodgepodge said that may be of interest to other people too 17:38:53 one is that we should probably quite soon explicitly add the ability to tag vantage points that goes beyond just the ASN 17:39:21 hodgepodge mentioned that it would be running on a machine that is a i2p relay, that may mean that the probe could receive different treatment and it needs to be taken into account I believe 17:39:40 ooni-probe supports a feature called "annotations" that allows you add to a report a key-value type data structure 17:40:03 and it was actually created exactly for this reason. To be able to say "this probe is running in a university" or "this probe is at office of company X" etc. 17:40:29 but from inspecting the reports nobody really seems to use it all that much, which makes sort of sense since it's not very well documented 17:41:52 hodgepodge: also as bnvk pointed out we will be working together on designing the API to the data and given your interest in that part of the OONI stack you may be interested in interacting more with us about that in the near future. 17:42:13 hodgepodge: can you remind me what information you are waiting on to connect to the backend services? 17:42:32 actually nevermind 17:42:46 it was the AWS API key with list cap? 17:44:26 While we're still here, and it isn't too late in your time zone, are there any additional steps I need to take to provision an ooni-pipeline environment apart from executing what is in ./bin (i.e. run Ansible, etc.)? 17:44:46 hellais: perhaps it beneficial to spec out ID codes for the annotations feature you mention, as seems like "filtering by University proboes" would be useful in dataviz side of things 17:45:20 I wonder if that can be inferred by ASN, since many universities are likely to run an ASN, if I'm not mistaken? 17:46:01 hodgepodge: by ooni-pipeline you mean the ng one? 17:46:21 The problem with a tagging approach would be that users would have to opt-in to letting us know that they are running a probe at a university. You could provide a mandatory menu similar to what OpenSSL does when you're generating a new cert? 17:46:43 bnvk: very good point 17:48:51 hodgepodge: regarding the privacy considerations, what you say is very true, however what it comes down to is if there is consent. The user should obviously have to opt-in for this, but if they are aware of the fact that this will end up being public we should not make it hard for them to do so 17:49:37 this would probably end up being part of the informed consent procedure we are developping with the harvard law school 17:50:05 That's wonderful. Glad to hear that you're working with them on this. 17:50:14 hodgepodge: what pipeline are you using? 17:50:18 you should be using the -ng one 17:50:25 the one without the -ng is abandonware 17:50:35 and I should probably rename it to -old or something 17:51:17 I haven't started with either pipeline yet, but I'll definitely use ooni-pipeline-ng going forward. 17:53:00 What is the best way to access the bridge reachability reports? I can filter through the reports using api.ooni.nu/api/reports, and filter by type? At the hackathon, I am going to be transforming the reports into a relational form with @hugg to allow for more interactive queries to be performed. 17:53:03 hodgepodge: ok, I would advise you to not focus too much on the old one and just inspect the code to see what the logic is 17:53:30 if you have some questions on some of the logic ask me and I will provide you with an answer as soon as possible 17:53:47 but it's probably not worth your trouble to figure out how to deploy the old pipline since it was quite a pain 17:56:26 (hopefully this isn't getting off-topic for the dev. discussions) 17:56:53 hodgepodge: in theory this should do the trick: http://api.ooni.io/api/reports/list?by=test_name 17:57:00 but I believe I just crashed the api for some reason 17:57:40 that should however give you in the keys the names of the tests and in the values the IDs and some other metadata about the report 17:58:21 hodgepodge: these are very fine questions, but time is tyrann so I would say we further this discussion out of badn 17:58:24 *band 17:58:54 Awesome, from there I think that I can use S3 to fetch the reports, and cache them locally. 17:59:00 hodgepodge: can you also send me a PGP signed email to remind me about the AWS token (I couldn't find an email from you about that in my inbox :( ) 17:59:29 so let's do a quick and compressed, juga style next steps 17:59:48 no need to request to speak, just type in one line, copy-paste - enter! 17:59:48 I agree, this would best be reserved for an OoB discussion. I'll send you an e-mail later today, as I don't have access to a machine with GPG right now. You should get it in the morning. 18:00:06 API fuzzing. Clearly. 18:00:12 ;) 18:01:05 Once the next iteration is underway, I can help with that. 18:03:04 I plan to work on finishing the Tor test for CCT. Doing more organisation and communication for the ADINA event. Helping out the people that are getting started working with OONI and pointing them in a direction. 18:04:12 now not juga style: 18:04:17 Todo: 18:04:17 * test seems to don't get any data from psiphon from stdout 18:04:17 * modify process.py in a different way 18:04:17 * figure out how the ssh process before test finish (right now modifying process.py, it kills the python process, but not the ssh process) 18:04:20 * solve the hardcoded relative paths in psi_client.py from the test. (Alternative: fork psiphon python client and ask upstream to merge?) 18:04:23 Further todo: 18:04:25 * talk with anadahz about the script to install psiphon 18:04:27 * if ssh already running, http request will success, but using other ssh process 18:04:30 * check if ssh to all psiphon servers is blocked 18:04:33 * Look other possible tasks in https://pad.riseup.net/p/s34qp95tNvcv, create tickets 18:05:23 We should add an i2p reachability test as well. I might be able to work with some of the developers in the associated community on that. 18:05:51 sbs: ? 18:05:52 anadahz: ? 18:05:54 landers: ? 18:05:55 bnvk: ? 18:07:04 finish ADINA preparation tasks 18:07:16 completer lepidopter release 18:07:35 reachout to all the people that would like to use the image 18:07:46 continue on the munin EOF 18:07:52 Todo: 18:08:41 * Start specing out REST API endpoints and data strutures with hellais 18:09:09 * Start on wireframes of OONI Explorer 18:09:12 EOF 18:09:57 I will attempt to finish the login of ADINA as soon as possible and then help with ADINA and possibly do something on the MK front. EOF 18:10:58 most excellent 18:11:05 thank you all for attending the gathering 18:11:15 appologies for some minutes of over time 18:11:35 but I think it makes sense given the number of people and importance of some matters 18:11:42 have fun! 18:11:44 #endmeeting 18:11:56 #endmeeting 18:11:59 oh noes 18:12:00 #help 18:12:14 the meetbot is not botting 18:12:25 will ping the devs 18:12:31 hellais: you need to change your username 18:12:40 hellais: you were hellais_ when you started it 18:13:29 #endmeeting