17:00:47 <hellais> #startmeeting OONI weekly gathering 2016-08-08
17:00:47 <MeetBot> Meeting started Mon Aug  8 17:00:47 2016 UTC.  The chair is hellais. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:00:47 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
17:00:50 <anadahz> hello
17:01:01 <agrabeli> hello
17:01:01 <hellais> hellow
17:01:11 <darkk> hi
17:02:07 <hellais> ok so let's get started
17:02:26 <hellais> #topic beginnin of ooniprobe 2.0.0 alpha testing
17:03:46 <landers> here
17:04:03 * sbs here
17:04:25 <sbs> (going to attend only the last 30', then I have to go)
17:05:09 <hellais> so we have been working on the new release of ooniprobe that features a web UI. Since this is introducing a lot of fundamental changes to how ooniprobe works, it's considered a major release and we are going to be doing a bit of testing before release it to our clients.
17:05:50 <hellais> I am going to send out to the mailing list an announcement to invite people to do alpha testing of it and report bugs they find in it, but I also wanted to bring that up here as well.
17:06:15 <hellais> You can find the release tagged here: https://github.com/TheTorProject/ooni-probe/releases/tag/v2.0.0-alpha.2 with some minimal setup instructions.
17:08:35 <hellais> anyways that is what I have to announce. Does anybody have any questions on this? Or has somebody already tried it out that would like to comment on it?
17:09:37 <agrabeli> thanks hellais. I tested it last week and it seemed to work well, except for the country code and ASN which were not detected (though there are tickets about that, I think)
17:09:59 <darkk> should it work with RPi & Raspbian as a platform?
17:10:02 <sbs> hellais: I have started to test it right now -- doing the wizard
17:10:25 <anadahz> hellais: apart from UX likes/dislikes do you have any specific area that you would prefer having more feedback?
17:11:36 <hellais> agrabeli: yes, the issues you reported with the IP address not being detected properly on first boot have since been fixed and are no longer present in alpha.2 (the one you tested was an even earlier alpha)
17:12:00 <hellais> darkk: yes, the plan is to have this be the web interface for the raspberry pi deployment of ooniprobe.
17:12:59 <agrabeli> hellais: great! I'm testing the alpha.2 now
17:13:21 <hellais> anadahz: well general stability issues and changes that break how you were using ooniprobe in the past.
17:13:51 <darkk> BTW, I had a lot of issues with power supplies while running RPi 24/7. Every power supply I used died in a couple of months, the only durable one was the B&N Nook charger. Have users reported anything alike?
17:15:21 <agrabeli> darkk: nope, none of our partners who are using Pis have reported anything of the sort (so far)...
17:15:46 <agrabeli> darkk: but yeah, this is definitely something worth checking.
17:16:03 <anadahz> darkk: We had no reports of power supplies failing yet. But this is a very common issue in the Raspberry Pi world, including SD cards and cables
17:16:04 <hellais> darkk: did you also have issues with the "official" rapsberry pi power supply?
17:16:55 <agrabeli> darkk: I also have a Pi running 24/7 and I haven't had any power supply issues...
17:17:23 <darkk> hellais: no, I've not tired "official" PSU, moreover, I doubt it existed when I ordered RPI1 back then several years ago. I should probably try to get "official" one for RPi3 I got lately.
17:18:37 <anadahz> hellais: Can you summarize the options that will no longer be available in ooniprobe version 2 (compared to older versions)?
17:19:21 <hellais> anadahz: it's not much that options are no longer going to be available, but rather that the behvaiour of ooniprobe itself changes a little bit.
17:21:06 <anadahz> will it be possible for ooniprobe to run in headless mode without using the web GUI?
17:21:40 <agrabeli> anadahz: how do you mean?
17:21:44 <hellais> anadahz: namely: 1) oonideckgen doesn't exist anymore, or rather it exists, but it really doesn't make much sense to run it more than once since the inputs are now parametrized in the deck it generates 2) ooniprobe by default doesn't write the report to your CWD, but it writes it to /var/lib/ooni or ~/.ooni/ whichever one you have write permissions to 3) ooniresources is now dropped in favour of handling upd
17:21:50 <hellais> ating of inputs via scheduled tasks, the command still exists, but it does nothing 4) Most users will just start the system daemon and forget
17:21:53 <sbs> anadahz: afaict yes
17:22:12 <hellais> anadahz: yeah you can still manually run tests with ooniprobe, but it's not going to be the recommended method of running it.
17:22:59 <sbs> hellais: when you run a test via command line, does it check whether there is a daemon running or could the two things run tests exactly at the same time?
17:23:57 <anadahz> there are a number of headless servers running ooniprobe, it will be nice if we could still support them
17:24:04 <hellais> sbs: currently ooniprobe and ooniprobe-agent don't cooperate with one another, so actually they could end up running the tests at the same time
17:24:33 <hellais> anadahz: yeah, these "headless servers" should run ooniprobe-agent. There is no reason for them to have cronjobs setup to run ooniprobe the cli
17:24:35 <MightyOctopus> [13ooni-probe] 15bassosimone pushed 1 new commit to 06v2.0.0-alpha: 02https://git.io/v6Gpr
17:24:35 <MightyOctopus> 13ooni-probe/06v2.0.0-alpha 148048dbd 15Simone Basso: settings.py: make sure default paths are okay
17:25:18 <hellais> ooniprobe the CLI is meant to be used by somebody that is interested in manually triggering tests, not for running on servers periodically.
17:25:52 <anadahz> so ooniprobe-agent can run with a specific test deck or this concept has been replaced by something else?
17:26:05 <hellais> I mean it was never really in the design goals of it to be instrumented through cron, we ended up doing that because it was a quick a dirty way of getting periodi measurements, but the ooniprobe-agent is specifically designed for this purpose
17:26:26 <hellais> anadahz: yes ooniprobe-agent has a set of available decks that you can configure from the web UI
17:26:49 <hellais> the deck format has actually changed and it now also includes scheduling information in them
17:27:28 <hellais> enabling a deck is a matter of creating a symlink between the /usr/share/ooni/decks-available and /var/lib/ooni/decks-enabled/
17:27:45 <hellais> this is how the full deck now looks like: https://github.com/TheTorProject/ooni-probe/blob/v2.0.0-alpha/data/decks/web-full.yaml
17:28:54 <anadahz> hellais: is ooniprobe-agent configurable without the need of the web GUI, thus can I programmatically setup and configure a probe in a headless server?
17:29:22 * sbs going
17:29:26 <sbs> bye!
17:30:41 <anadahz> ah i see so ooniprobe-agent polls /var/lib/ooni/decks-enabled/ every XXX amount of time for changes?
17:31:35 <hellais> anadahz: yes, you can though, it's not something that we are going to be supporting "officially", mainly because we want people to agree to go through the informed consent procedure before running it.
17:32:44 <hellais> that is we want them to connect to the web UI and go through the first start wizard to read the risks document, answer the quiz and do the first setup.
17:33:32 <hellais> overriding this is quite simple (it's a matter of dropping a certain file in a certain location), but it's not something that is going to be documented.
17:34:13 <anadahz> hellais: I guess exposing the ooniprobe web GUI in public IP space will pose some security issues for server running at the internetz?
17:37:13 <hellais> yeah, you should probably only expose it as a Tor hidden service
17:38:00 <anadahz> I'm just considered about headless ooniprobe setups as I and a couple of other people that run a similar will find this process of configuring ooniprobe via GUI a very tedious task..
17:38:06 <darkk> [✓] I accept terms of the legal agreement. Do we consider RIPE-Atlas style when the web UI is hosted at controller-side and the probe is just an opaque box? IMHO the only unavoidable reason to have a web UI at the probe is to have a control channel when the Manager UI is blocked by malicious cable guy.
17:39:12 <meejah> https://github.com/warner/petmail/blob/master/docs/steal_this_software.md <-- how to launch a Web UI for a local thing securely
17:42:32 <landers> meejah: dayumn
17:43:46 <hellais> anadahz: you can also accept the informed consent procedure through the CLI, but it's also something that requires interaction.
17:45:39 <hellais> darkk: What do you mean by "RIPE-Atlas style"? You mean having some system to remotely control your probe and other peoples probes?
17:45:53 <hellais> meejah: thanks for the pointer, that looks like a worthwhile read.
17:47:28 <darkk> hellais: exactly. IIRC, we were discussing alike controller some time ago.
17:49:24 <anadahz> another potential issue if ooniprobe web GUI is being exposed in all network interfaces is that everyone on the LAN or the internet (if the user is on public facing IP) can manipulate ooniprobe configuration, start/stop measurements, etc...
17:51:07 <landers> anadahz: meejah's link talks about how to auth only the local browser
17:52:22 <hellais> darkk: yeah, that is something that we have planned, but is going to be separate from the Web UI and I am not yet sure if we want to have full remote control capabilities via our orchestration mechanism, but probably just be able to say "if you are configured to run this type of test, use these inputs or if you have a policy that allows it run also this type of measurements". It's anyways something that need
17:52:29 <hellais> s to be speced out properly.
17:53:02 <anadahz> landers: i guess this require some time to be deployed that most probably will not be ready before the stable release
17:53:50 <landers> ETA for stable?
17:54:03 <darkk> on the other hand, it may be a good thing to be able to start measurement against blocked https://bitter.com when half-of-internetz are shut down for you and be able to upload the data to the pipeline afterwards
17:59:02 <hellais> anadahz: by default it should only bind to localhost (though I realise now that it's not actually doing this, but it will be changed) as for how it should work on the pis, it's already the case that the default image comes with insecure defaults (i.e. ssh open on port 22 with a default password) so probably we should be integrating into the setup process some sort of re-keying mechanism for both ssh and the
17:59:08 <hellais> web UI.
18:02:19 <hellais> landers: we are aiming to release 2.0.0 to clients by the beginning of September.
18:02:46 <anadahz> hellais: so from ooniprobe version 2 the user will be responsible for enabling the measurements in lepidopter and by default lepidopter will be doing nothing unless the user sets the device?
18:03:20 <anadahz> s/device/ooniprobe web GUI
18:06:10 <hellais> correct
18:06:18 <hellais> that will be the default out-of-the-box behaviour
18:07:15 <anadahz> and I guess this will also affect other lepidopter features such as auto-updates and remote management?
18:08:01 <hellais> I think auto-updates should always happen by default
18:08:18 <hellais> remote management probably is one of those things that we want to enable on a case by case basis
18:08:29 <anadahz> I guess if the user hasn't setup ooniprobe there is no sense of performing auto-updates or other operations
18:09:34 <hellais> well we still want them to have the latest version of the software on it even if they haven't yet set it up
18:10:38 <hellais> I mean it's not uncommon for a person to recieve the pi after a new release has been done and we would like to have it update even if they haven't yet taken the time to connect to it and configure it
18:11:57 <anadahz> This can be feasible in ethernet cable deployments only...
18:12:45 <anadahz> In hostAP mode there will be no internet connectivity
18:13:12 <hellais> yeah, but we don't currently support those types of deployments anyways
18:13:26 <hellais> and I am also a bit unsure if we want to be supporting them at all
18:13:30 <darkk> are updates signed? is `pip` used for production updates?
18:13:46 <agrabeli> Based on experience with partners' needs so far, I think it's vital that we have auto-updates since most of our users wouldn't be able to manually update them anyway
18:14:06 <darkk> what should be the probe measuring if it has no internet connectivity?..
18:15:38 <hellais> darkk: debian/ubuntu packages are signed and verified. pip packages are also signed, though pip install doesn't verify signatures when it downloads packages.
18:16:12 <hellais> darkk: I believe the raspberry pi image currently uses pip to automatically update ooniprobe and it's dependencies, so it's only reliant on https.
18:16:42 <anadahz> darkk: if you have no access on an ethernet outlet there are no many things to do (except from buying an access point or similar devices) to have internet connectivity in a RasPi
18:17:17 <darkk> RPi3 has some wireless IIRC.
18:17:44 <elation1> rpi3 does have a wireless chip
18:17:45 <darkk> hellais: https should be ok. I was a bit afraid of unattended pip update via http though :)
18:17:56 <agrabeli> darkk: but Lepidopter doesn't (currently) support wifi
18:17:59 <anadahz> elation1: darkk: Yes but how can a user connect to set it up?
18:18:45 <elation1> I don't believe that it has a default AP setup. But, that can be done when building the image.
18:19:59 <anadahz> elation1: exactly, but this will still require user intervention and the Pi will be without internet connectivity until the user setups the device
18:20:08 <hellais> darkk: yeah, I think we also make sure we are using a recentish version of pip that actually does cert validation properly (some older pips did this terrible thing of doing a TLS handshake over one socket verifying the cert and then downloading the package over a new socket :P)
18:20:33 <darkk> anadahz: good question. Bootstrapping depends on user ability (e.g. connect Pi via p2p ethernet to laptop and configure wifi), agrabeli has more info regarding users' hardware.
18:22:22 <darkk> the pi also has bluetooth, so we can do something to manage pi from smartphone if user has no PC or his laptop has no ethernet (like macbook) :)
18:22:54 <darkk> but I don't know if there are such "bounded" users out there.
18:22:58 <anadahz> darkk: i find this a bit more complicated depending on the OS for a user than the host AP functionality for the wireless
18:23:47 <anadahz> if the user connect the RasPi via ethernet cable the host AP mode should be disabled
18:27:22 <hellais> anadahz: can you create an access point with hostapd and at the same time connect to another access point?
18:27:43 <anadahz> There are many use cases out there but I guess finding an available ethernet port is not that easy.
18:29:27 <anadahz> hellais: I'm not sure if this is possible with the RPi3 wireless device
18:30:23 <darkk> anadahz: yep, network-manager-over-bluetooth does not look like a real way to go, I'm mostly kidding.
18:31:19 <anadahz> Will be totally feasible with an additional wireless interface (eg: a USB dongle)
18:33:00 <anadahz> hellais: but which access point are you planning to setup as default?
18:33:21 <hellais> hum I don't think requiring extra hardware is doable.
18:33:59 <hellais> anadahz: well I was thinking that in the hostapd setup you would spin up a captive portal AP that the user logs into and then from there configured the access details of their wifi network.
18:34:26 <darkk> do we have a user-story for this wireless manipulations or is it just greenfield discussion?
18:34:40 <hellais> however this only works if a single card can act as both AP and client. Doing a bit of cursory searching it seems like certain devices support this mode of operation, but they can do it only if it's on the same channel.
18:36:03 <hellais> darkk: yeah, actually there is no story for this and I think the takeaway from this discussion is that doing this is going to be quite tricky and we don't really have that much time to invest in this.
18:36:35 <anadahz> hellais: I guess then you mean wireless network scanning rather than establishing a connection to another AP and on the same time serve a captive portal on a different network?
18:38:48 <hellais> anadahz: no I mean the later.
18:39:21 <darkk> maybe that's not that tricky but I'd rather wait till the first partner with no-ethernet issues appears.
18:39:51 <agrabeli> darkk: hehe indeed
18:41:27 <hellais> darkk: +1
18:41:36 <agrabeli> in general, if we need to talk about priorities in terms of meeting partners' needs, I would sum them up to the following: (1) auto-updates for sure, (2) as having remote access to them so that we can help trouble-shoot (for example, with tor hidden services), (3) ensuring that the distribution includes bridges, in the instance that Tor is blocked in their network...
18:41:58 <anadahz> hellais: anyway usually in captive portals you don't need to be connected in two networks, upon configuration of the network that the RasPi is supposed to connect you are being connected and wait in the same for the page to load (until the connection is established).
18:42:55 <agrabeli> It's also important I think that partners are required to read and agree to 'informed consent' documentation, when setting up their Pis
18:43:00 <hellais> anadahz: yeah, but if the wifi network of the captive portal goes down, the client will get booted from it and the page will never load because the network is now offline.
18:44:02 <anadahz> agrabeli: We should also be thinking of users that are not partners and have no alternative way of running ooniprobe...
18:44:37 <hellais> speaking of reading the informed consent procedure when setting up the pi, we should think of how people are going to discover the local address of the raspberry pi
18:44:49 <agrabeli> Other partner needs that I would add include enabling them to choose which tests to run (though I guess this will be solved as part of the orchestration?), as well as which URLs to test, and which ones not to test (though this part might be tricky)
18:45:08 <hellais> anadahz: I think for them it's even more important that they read the informed consent documentation, since they have not even been briefed by us on the risks.
18:45:28 <agrabeli> anadahz: sure, though if someone is willing or able to get a RasPi, I'm pretty sure they can get an ethernet cable too ;)
18:45:57 <anadahz> hellais: the practice is similar of setting up any router there is an amount of time that the page "waits" prior to redirection
18:47:21 <agrabeli> hellais: indeed! No I think it's vital that users are required to consent to the documentation as a prerequisite to using Lepidopter. Allowing users to run Lepidopter without requiring them to read about risks could be viewed as unethical from our side.
18:47:48 <anadahz> we could also have a message in case the user for any reason loses this page (like accidental browser exit)
18:47:52 <hellais> anadahz: yes, but if you are connected to the capitive portal on ip 192.168.1.1 on wifi network with SSID: X and then you tell it connect to network sid SSID: Y with these credentials and it needs to tear down network X, the client that is connected to page on 192.168.1.1 will be disconnected and may perhaps reconnect automatically to network Y (if they had it already configured), but most probably the IP of
18:47:58 <hellais> the captive portal will have changed.
18:48:00 <agrabeli> and it's something we have committed to do anyway, as part of our deliverables, so I believe that is non-negotiable
18:49:57 <hellais> anyways it seems like we are way overtime at this point, so unless there are some final remarks or considerations I would say we can end this here
18:51:12 <anadahz> hellais: The user will not be presented with an IP adress but rather with a hostname if the RasPi is already connected to the network (if the credential are corrects) then the page will redirect propely on the same hostname as before.
18:52:40 <hellais> anadahz: that will only work if the network B sets up DNS records for the hostname of the raspberry pi and in any case only if the user is using the canonical resolver of the router.
18:53:32 <anadahz> hellais: which is 90% of the cases
18:54:20 <darkk> I doubt the numbers, the only SOHO network that set up DNS records for LAN clients was OpenWRT-based box...
18:54:35 <darkk> s/only/only SOHO network I used/
18:55:08 <anadahz> I guess the users will be able find the IP/hostname of lepidopter like when they are using ethernet connectivity
18:55:26 <hellais> yeah I also haven't encountered that many home routers that setup the DNS records for land clients...
18:56:34 <hellais> anyways futher research should be done on this topic
18:56:46 <anadahz> hellais: darkk: I have never encounter home routers that don't assign DNS records for clients
18:57:38 <hellais> I think not so much on the wifi captive portal AP setup thing, since that is not amongst our priorities, but this topic of figure out the IP of the pi is something that is quite important and we should have some reliable way of doing it
18:58:19 <darkk> L2 autodiscovery may be another useful keyword for the task, I mean avahi/bonjour/zeroconf. But I don't know how Mac/Win/ios/Android vs avahi/bonjour support matrix looks like.
18:58:38 <hellais> darkk: yeah, that was what I way suggesting for it actually.
18:59:23 <hellais> The only issue is that UIs for it are a bit inconsistent, but I think we can overcome that with good documentation that includes screenshots, or make a minimal connector for it
18:59:40 <anadahz> re: https://github.com/TheTorProject/lepidopter/issues/46
19:00:13 <hellais> anadahz: ah yeah, I thought we had had this discussion already :P
19:00:23 <anadahz> :)
19:00:38 <elation1> I have built wireless web-based setup bootstrapping UI's for two projects (commotion and copilot) If you would like I can see if I can't find the process documentation for either and push them here so you can see some other groups back and forths.
19:01:21 <anadahz> elation1: that would be awesome!
19:01:34 <hellais> elation1: yes, that would be quite useful I think
19:06:19 <anadahz> I guess at this point and since we are already very late we can end this meeting, if anyone has nothing to share with us. :)
19:07:59 <hellais> ok sounds good
19:08:08 <hellais> thank you all for attending and for the interesting discussions
19:08:09 <hellais> #endmeeting