17:15:32 #startmeeting snowflake meeting 17:15:32 Meeting started Thu Apr 12 17:15:32 2018 UTC. The chair is asn. Information about MeetBot at http://wiki.debian.org/MeetBot. 17:15:32 Useful Commands: #action #agreed #help #info #idea #link #topic. 17:15:39 arlolra: ok. i just made a note at the end of the agenda, to think about what a part-time arlo job might look like 17:15:52 alrighty 17:16:14 ok. item 1, how are we doing at the tasks we identified from last time as needing doing? 17:16:32 the snowflake.torproject.org website is all set and ready i think. let me know if that isn't so. 17:16:38 sukhe, any news on the windows build front? 17:16:54 ok I can go first 17:16:59 great 17:17:10 Yes, snowflake.torproject.org is working, and I'm planning to move content there. 17:17:21 so there is progress, but the builds are not ready yet. to be clear, I am mostly building on all the work dcf1 has already done 17:17:28 Longer term, it will be nice to do a style guide-ificiation, like isabela said 17:17:59 so the webrtc builds on Windows are done (again, dcf1, arlolra) and now we are trying to build go-webrtc 17:18:37 there is a linking error, which we think is due to C++ ABI (webrtc is compiled with clang and go-webrtc with mingw-w64) 17:18:46 arma4 sukhe dcf1 are you going to add more content at snowflake.torproject.org? 17:18:53 (This is a place where cgo actually is causing a problem, because webrtc wants to compile with clang and cgo wants to use gcc, and they have incompatible ABIs on Windows) 17:18:54 dcf1: if you want our help ( antonela help) 17:18:56 I'd like to provide the branded version 17:18:59 yes 17:19:02 ^^ this :) 17:19:08 yes 17:19:16 I am currently trying to fix that. it's difficult to come up with an ETA 17:19:19 dcf1: if you want you can provide content and anto can organize it in a design 17:19:43 antonela: I will let dcf1 and arlolra answer that one :) 17:19:43 dcf1: sukhe: and what's the fix wrt cgo webrtc clang/gcc? 17:19:51 sukhe :) 17:19:53 antonela: Initially it will just be a copy of https://keroserene.net/snowflake/ (that's where we're migrating the page from) 17:20:06 dcf1: yes, i have it saved 17:20:11 all it? 17:20:34 asn: either fixing the linking errors or even better, build go-webrtc with clang but that has problems as well 17:20:49 because cgo wants to use gcc, like dcf1 mentioned 17:21:13 sukhe: ok. is there anything we can do, in terms of people or etc, to help you there? or is the answer just that you need to beat your head against it for a while more? 17:21:15 antonela: yes, at first. The important part is not the text in the page, it's the JavaScript code. We need to move the JavaScript code there in order to to other tasks. 17:21:15 dcf1: should I focus on https://github.com/keroserene/go-webrtc/pull/69 (the wrapper) or https://github.com/azadi/tor-browser-build/commits/go-webrtc ? 17:21:22 dcf1: what is needed to get go-webrtc compiled with a/the clang toolchain? 17:21:37 sukhe: I have no clue :) I'm at the same impasse you are. 17:21:46 arma4: I think it's mostly that unless I am following the wrong approach but to prevent that from happening, I am keeping dcf1 and arlolra updated 17:22:15 GeKo: not sure what's required. I think that it is not actually supported by Go, as Hooman found out. He found a GitHub issue, let me find it. 17:22:30 arma4: I plan to ask "the experts" since I have tried what I could so that should help 17:22:30 because the medium term plan is to move to a clang/mingw-w64 toolchain anyway 17:22:44 because firefox needs it 17:23:06 sukhe: right, bringing in other people seems like a smart move. if you find that we need to ask the twitters for help, for example, steph can totally do that. maybe there are less nuclear options too. 17:23:07 GeKo: https://github.com/golang/go/issues/17014 and https://github.com/golang/go/issues/20982 are the issues with cgo but I haven't read them in detail 17:23:14 GeKo: yes, agreed on clang/mingw-w64. 17:23:20 dcf1: cool! I can work with this content. Thanks! 17:23:32 ok, thanks 17:24:22 arma4: ssup with the item "- arma: figure out cookies w/ weasel" 17:24:28 isabela: it's done 17:24:31 all resolved 17:25:00 * arma4 is hunting through last week's pad to try to notice other things we were going to do but have missed (if you know any too, let us know) 17:25:07 Right, sukhe's https://github.com/golang/go/issues/17014 is the issue I was looking for, clang for cgo is not a supported configuration on windows; it's unclear how much work it will be to make it work. 17:25:09 cool 17:25:38 so yeah, that's it from me, hoping to resolve this asap 17:25:48 sukhe: ack thx! 17:25:52 (kept some notes on pad) 17:26:09 dcf1: yeah, also related is why I didn't try the other stuff (like removing win_sdk from the webrtc builds) because I wanted us to have a working build first, then the enhacements 17:26:13 great. we know this is a hard one, so please bring in whatever help you think you need, and/or escalate to us to try to get you that help. 17:26:13 #23947 17:26:21 ^^ ssup with this 17:26:27 i dont think is in the roadmap 17:26:34 isabela: that ticket is all done, and the remaining thing is "populate snowflake.tp.o website" 17:26:55 and i guess part of populating the website is that they need to move the proxy code over to the new website 17:26:58 ok 17:26:59 and point new snowflakes at it 17:27:06 does that involve a tor browser update too? 17:27:07 * isabela is digging 17:27:22 GeKo: ^ 17:27:27 i think maybe no? since snowflake clients just talk to the broker? 17:27:32 arma4: yes, it will require a change in the client code and a new browser update. 17:27:44 ok. why? 17:27:45 But it's graceful, old versions will continue to work under the old location. 17:27:51 still trying to get the right mental model here :) 17:28:12 It's in one of Tor Browser's config files 17:28:38 don't snowflake clients (a) tell the broker they need a snowflake, and then (b) await a snowflake contacting them? 17:28:45 https://gitweb.torproject.org/builders/tor-browser-build.git/tree/projects/tor-browser/Bundle-Data/PTConfigs/linux/torrc-defaults-appendix 17:28:56 "-url https://snowflake-reg.appspot.com/" that part needs to change 17:29:13 ah 17:29:20 arma4: oh wait, I might be confusing something. 17:29:35 https://trac.torproject.org/projects/tor/wiki/doc/Snowflake#Tickets 17:29:53 see the graph 17:29:53 arma4: okay yes, you are right, for just moveing from keroserene.net to snowflake.tpo, we don't need a client update 17:30:09 whew. mental model remains intact 17:30:12 we're ultimately trying to change the broker 17:30:13 who controls https://snowflake-reg.appspot.com/? is that also an issue in the long run? 17:30:18 but part of the reason for moving from keroserene.net is to start using our standalone (apart from APp Engine) broker, and that is the part that needs a client update 17:30:28 ah, okay 17:30:40 mcs: serene controls https://snowflake-reg.appspot.com/, which is why we're trying to stop using it. 17:30:53 so the glorious future involves running a broker not actually on appspot, and then we can use appspot to front to it, but also we can use other things to front to it? 17:31:07 arma4: correct 17:31:10 great 17:31:28 which ticket is "move the broker to a new more standalone location"? 17:31:33 In fact https://snowflake-reg-test.appspot.com/ (which is mine) is already fronting for https://snowflake-broker.bamsoftware.com/, which is the standalone broker. 17:31:47 #22874 17:31:54 okeydoke 17:32:13 22 mins. were there other items from last week that we should report progress on? or, on to next items? 17:32:13 Yeah see my comment:7 "I think we're not going to regain access to ​https://snowflake-reg.appspot.com/. I think the way forward is to do #23947; i.e., move the proxy-hosting page away from keroserene.net, and then we configure the proxy on the new host to use the new broker." 17:32:37 sounds good. all that remains is to just do the things. 17:33:08 so, my item 2 on the agenda. i wanted to understand better our plans for the arms race: 17:33:23 perhaps we should ask t0mmy what kind of specific info they need for drafting the job posting? 17:33:26 based on "t0mmy: started on a job description but need specific info 17:33:36 * arma4 backs up 17:33:37 yes ok great 17:33:43 oh no rush 17:34:04 the roadmap under (4) 17:34:30 what about it :) 17:34:34 is this what I should include in a dev job description? 17:34:44 these tasks, I mean 17:34:49 ah hm. i think no. 17:34:49 no 17:34:56 i think these immediate tasks that we know about, we have a pretty good handle on 17:35:05 lets look at the pad 17:35:08 some of them will be open-ended, like "keep x working" or "make x more robust" 17:35:27 https://pad.riseup.net/p/K4cF5vPEU7Rg-censorshipteam-jobposts 17:35:31 but what i'm imagining for the snowflake dev is that their main job is to keep snowflake actually working in, say, china 17:35:34 we need to write skills 17:35:39 so must have and nice to have 17:35:41 and that means being on top of censorship that happens 17:35:45 not tasks 17:35:49 arma4 right so it's a reactive position 17:35:50 i mean we can list some tasks 17:35:55 isabela aha, filling this out would be super 17:35:58 but general ones that they will be executing at the job 17:36:02 yep. skills and abilities. 17:36:20 i agree we should fill out this list for the job descriptions. i think we won't succeed at doing it during this meeting though. 17:36:33 We could use some better monitoring/metrics, something automated from the broker 17:36:38 building that could be part of the job 17:36:58 arma4: agreed 17:37:20 ok. two things to do for the future meetings: (a) actually get a handle on all the known tasks and try to organize them into a roadmap, 17:37:21 Another necessary ongoing skill is coping with upstream changes to the webrtc library 17:37:22 * asn adds it to action items in the end 17:37:33 and (b) flesh out the two job description visions 17:37:43 It doesn't have a stable API and it's an unpredictable amount of work when there's a new Chromium release. 17:37:50 dcf1: ah ha, right, the surprises won't just come from the censor, they will come from chrome too. 17:37:50 but we can paste there what was said so far 17:37:53 okay, so I'll pause on the job descriptions for now? 17:37:54 we dont need to loose it :) 17:38:00 isabela: yes please 17:38:39 An OONI test for WebRTC could be useful, though there are many details to consider. Let me find a opst. 17:39:03 dcf1: ^^ this 17:39:12 we can add as taks for ooni at the other spreadsheet 17:39:17 k 17:39:18 https://lists.torproject.org/pipermail/ooni-dev/2017-March/000496.html 17:39:32 * isabela leaves the job description pad alone :) 17:39:42 * t0mmy does likewise 17:39:43 yes. if we want a dev that responds to incidents, then we need a way to detect incidents. otherwise the dev will just sit around and wander. 17:39:44 t0mmy: please bug people over the week so folks dont forget about it :) 17:39:54 ok. second try at my item 2 from the agenda. :) as i understand it, we use the data channel in webrtc. and not much else does -- hangouts does not, for example. and i think it's possible/easy to tell on the wire whether it's using the data channel or the other channels. 17:40:00 isabela ah yeah, we can work btw meetings 17:40:01 what was our vision for how that arms race proceeds? 17:40:05 t0mmy: also we should try to bug all the original thread again with updates on these meetings and these tasks 17:40:07 * dcf1 is trying to help with job description, by brainstorming chronic maintenance tasks 17:40:13 isabela +1 17:40:19 because the bad version of the vision is that we roll this out, and then censor blocks webrtc data channel, and then oops. 17:40:20 dcf1: :) 17:40:21 will do 17:40:34 t0mmy: tx 17:41:11 i think dcf1 and arlolra only can answer the future of arms race discussion 17:41:15 For some reason the rest of that OONI/WebRTC thread is not linked 17:41:16 https://lists.torproject.org/pipermail/ooni-dev/2017-March/000497.html 17:41:26 i can just shrug and say "let's move from data channel of webrtc, to the one that hangouts uses" 17:41:49 asn: if we just spent a bunch of work ripping out the parts of webrtc that aren't the data channel, though, ... 17:41:58 asn: it's not quite that easy 17:42:25 arma4: I did that already in https://trac.torproject.org/projects/tor/ticket/19569#comment:1 17:42:52 Detectability of data channel vs AV channel is something that could be tested for with Adversary Lab. Alternatively, you could have someone dissect packet traces. 17:44:02 but yes, dcf1 and arlolra are the ones to answer this one. is there a plan already planned? or was it "step one, make the data channel work"? 17:44:17 I think using the AV channel for application data will be some significant work. 17:44:20 asn: Hangouts doesn't use a plain data channel or plain media channel, but something else custom (last time I checked) 17:44:37 ouch 17:45:07 asn: https://arxiv.org/pdf/1605.08805v1.pdf 17:45:09 is hangouts our main target in terms of hiding behind it? 17:45:11 "Some WebRTC applications that use 17:45:11 SRTP make use of an older type of key exchange called 17:45:14 SDES [2]—in this case no DTLS handshake occurs." 17:45:19 Hangouts is one such application, afaik 17:45:24 hello paper 17:45:44 Oh what a nice paper. 17:46:05 asn, you might also like https://www.bamsoftware.com/papers/thesis/#sec:webrtc-fingerprinting 17:46:11 +1 17:46:32 ok, so those look like they argue it's a hard problem 17:46:50 arma4: maybe not so much hard, as unknown at this point 17:46:59 quantifying how hard may actually be the hard problem :) 17:47:05 dcf1: can you paint a picture for us, briefly, about how you imagined the next steps going? 17:47:21 arma4: or censor-circumventor interaction? 17:47:32 because if we're pretty sure we're going to need to do the future steps, no time like the present to think them through 17:47:39 yes. of "they block this so we move to that" 17:48:22 Likely first step (arlolra check my intuition) is the censor blocks the STUN server 17:48:39 https://gitweb.torproject.org/builders/tor-browser-build.git/tree/projects/tor-browser/Bundle-Data/PTConfigs/linux/torrc-defaults-appendix 17:48:47 "-ice stun:stun.l.google.com:19302" that one 17:48:52 poor google 17:49:02 (so sad) 17:49:23 We haven't put a whole lot of thought into the STUN server, whether the one we've chosen is actually hard to block, what other applications use it, etc. 17:49:49 It is also the case that you can use any old STUN server, doesn't have to be a specific one--though of course if you have to change frequently 17:49:59 you want something more agile than a torrc file in Tor Browser 17:50:38 what would be a more agile solution? 17:50:52 I see that as the most obvious step for some engineer who gets the instruction to analyze and block it 17:51:00 load stuff off the network? but then they can block that? 17:51:23 if we assume domain fronting continues to work, then we can use it for low bandwidth stuff like getting a fresh stun server suggestion 17:51:32 Samdney: I don't know, a local database of many servers, maybe there's some kind of local client autodetection we can do 17:51:44 arma4: yes, potentially. 17:52:16 arma4: also I'm jazzed about an idea to use DNS-over-HTTPS for this kind of rendezvous information: https://trac.torproject.org/projects/tor/ticket/25594#comment:1 17:52:36 fun. like cloudflare's 1111 17:52:49 yeah 17:53:18 ok. imagine we solve the stun server question. what then? 17:53:35 So another likely next step, as I see it, is the aforementioned fingerprinting of the WebRTC implementation, or snowflake's use of data channels (the Fifield-Gil Epner tech report) 17:54:27 That one, it's hard to say how it will go, because so much depends on what kind of WebRTC exists in the wild 17:54:46 * arlolra tacitly trusts dcf1's intuition on all this :) 17:54:54 We don't have a https://tlsfingerprint.io/ for WebRTC. 17:55:24 https://tlsfingerprint.io/top/ is maybe more illustrative of what we would love to have 17:55:47 and, from the other side, what the censor would love to have too, for their own network 17:56:09 Another thing that is hard to predict, is how diverse the Snowflake proxy IP addresses will actually be 17:56:12 arma4: yes, true 17:56:31 ah ha. maybe we have 100 snowflakes and they just get blocked, and nobody else cares. 17:56:42 or, maybe they're all on comcast, so comcast gets blocked 17:56:54 Maybe we only have a few hundred, and maybe they are mostly all static proxy-go or Cupcake isntances, and they just die from incremental blocking over time. 17:57:09 in theory they can do multi-characteristic blocking too, like "comcast and stun" or "comcast and webrtc" 17:57:18 Yes, the nature of the AS is also a consideration 17:57:38 But: looking at it another way, you have basically the same situation with normal Tor bridges, 17:57:51 but it's easier to setup snowflake proxies than bridges right? 17:57:54 excapt that it's orders of magnitude easier to start a snowflake proxy 17:58:02 right 17:58:03 heh 17:58:06 (Maybe not easier to actually get people to *keep* them running, though) 17:58:13 true 17:58:46 So if you make a blog post, maybe you get thousands of new Snowflakes, whereas you might get dozens of new bridges, because people don't have to rent a VPS or whatever. 17:58:46 seems like someone needs to setup a community of snowflake proxies too. like with blog posts and stuff. not sure if that's the job of the developer or the PM. 17:59:06 ok. so let's say webrtc data channels are considered by the censor to be an acceptable loss. 17:59:22 asn: it's the community outreach team 17:59:26 * asn needs to go in a bit. i still have #endmeeting rights. 18:00:17 I shudder to mention it,but without data channels you could still conceivably encode a reliable stream into audio/video. 18:00:33 Like https://censorbib.nymity.ch/#McPherson2016a https://censorbib.nymity.ch/#Houmansadr2013a https://censorbib.nymity.ch/#Barradas2017a 18:00:42 arma4: you want me to #endmeeting, and you start it up again? 18:00:44 ok. the remaining items i was hoping to get to are: (a) more understanding of the arms race roadmap. (b) somebody to assure me that all hope isn't lost for a tor browser snowflake on android, and (c) we should decide if we like thursday meetings 18:00:46 before we wrap up -- is everyone clear on next steps? can we assign the action item tasks to people/groups? does it suit/make sense to meet next Thursday at this time again? 18:00:58 oh sorry, arma4, go for it 18:01:30 There is probably more low-hanging fruit in terms of DPI blocking of snowflake until you get to blocking all data channel traffic. 18:01:33 it sounds like we should keep chatting again on the arms race roadmap question. because i am nervous that we don't have it mapped out very far. 18:01:55 yeah, we didn't talk about all the fun ways to mess with the broker 18:01:55 blanu: right, meaning, they block on a particular characteristic, so we change that characteristic, and there's some back and forth for a while 18:01:55 I don't have very many ideas beyond what we've discussed 18:02:01 arma4: i think we can answer some of these questions next meeting 18:02:33 ok. sounds like we need a censorship arms race task force. maybe once we have more experts in our group. 18:02:38 maybe we need to 1. agree on the next meeting 2. send a call to it listing these questions and the other tasks we spoke of 18:02:45 like the job description pad 18:02:50 re: android, we have an arm build as a start (https://github.com/keroserene/go-webrtc/blob/master/lib/libwebrtc-linux-arm-magic.a) 18:02:59 great 18:03:08 arlolra: oh good, so it is a thing that we are hoping will work. great. 18:03:26 ok guys im off. i #endmeeting. sorry about that! keep it up! 18:03:28 #endmeeting