15:59:15 <meskio> #startmeeting tor anti-censorship meeting
15:59:15 <MeetBot> Meeting started Thu Feb 17 15:59:15 2022 UTC.  The chair is meskio. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:59:15 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
15:59:20 <meskio> hello everybody!
15:59:24 <meskio> here is our meeting pad: https://pad.riseup.net/p/tor-anti-censorship-keep
15:59:35 <meskio> feel free to add what you are working on and put items in the agenda
15:59:45 <cohosh> hi!
16:00:47 <meskio> we have one point to discuss: obfs4proxy-0.0.12 causing connection failure warnings, before eventually succeeding
16:00:58 <meskio> https://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/40804
16:01:14 * meskio is reading the issue, but please jump in if you know about it
16:01:38 <dcf1> This is something new with the upgrade to obfs4proxy-0.0.12
16:01:58 <shelikhoo> Hi~
16:02:06 <dcf1> 0.0.12 has a change to fix a distinguisher in the elligator encoding of public keys during the obfs4 handshake
16:02:29 <dcf1> regrettably I still haven't investigated to understand exactly what the nature of this distinguisher is
16:03:08 <dcf1> but since then, it seems all obfs4 bridge users have been seeing "unable to connect to X.X.X.X:443 ("general SOCKS server failure")" in the tor log, sometimes multiple times, before getting a working connection
16:03:38 <meskio> uff :(
16:03:42 <dcf1> the change in obfs4proxy-0.0.12 is supposed to be backward compatible (I can find documentation saying so in a second), but perhaps it is not fully
16:04:18 <shelikhoo> Will this error go away if we upgrade both client and server to the same version?
16:04:44 <cohosh> so it succeeds eventually?
16:04:45 <meskio> that will be tricky, to get every bridge operator to upgrade
16:04:50 <dcf1> my guess: obfs4proxy-0.0.12 client with an older server has something like a 50% probability of handshake failure and causing "general SOCKS server failure"), so you get an expected 2 tried before a connection works (geometrically distributed)
16:05:02 <cohosh> ah
16:06:18 <dcf1> Although I haven't looked into the elligator implementation problem, I suspect it is of a similar nature: actually indistinguishable some fixed fraction of the time, distinguishable with some probably the remainder of the time
16:07:15 <dcf1> it's killing me that there's not a general technical understanding of what the bug actually is, and I think I or someone is going to have to actually figure it out
16:07:33 <dcf1> shelikhoo: I don't know if upgrading the server fixes it
16:07:37 <meskio> AFAIK currently censors are not doing probabilistic attacks, so we should be fine to roll back if needed
16:07:42 <meskio> but we should fix that problem
16:08:10 <dcf1> cohosh: it seems like it succeeds eventually; I don't know exactly what mechanism causes tor to rety the bridge
16:08:13 <meskio> I have in my queue of things to do to get my hands dirty with obfs4, I could give it a try to this problem
16:08:22 <irl> https://packages.debian.org/sid/obfs4proxy 0.0.13 is now in debian
16:08:25 <meskio> but I have to recognize that I had no idea what I'm getting into
16:08:43 <meskio> irl: I doubt 0.0.13 fixes the problem, it just removes utls
16:08:56 <meskio> BTW, we might not want to upgrade to 0.0.13 in TB
16:09:05 <irl> right, if there is more than has to be fixed there then we should figure out what it is
16:09:13 <irl> and make sure that the fix is also going into debian
16:09:28 <dcf1> one of the difficulties is that the academic paper that introduces elligator is notoriously hard to read and understand
16:09:28 <meskio> sure, first figure out what is needed there
16:09:34 <arma2> dcf1: re the elligator bug, we talked to aaron johnson of nrl, who seems to have a handle on it
16:09:39 <arma2> i can introduce you if that would be useful
16:10:00 <arma2> he told us that indistinguishability is reduced by a factor of 8. and two factors of 8 if both sides haven't upgraded.
16:10:19 <dcf1> Loup Valliant (who worked with yawning to make the 0.0.12 fix) has made blog posts that are a lot more readable
16:10:29 <dcf1> https://loup-vaillant.fr/tutorials/cofactor about the bug specifically I believe
16:10:38 <dcf1> https://loup-vaillant.fr/articles/implementing-elligator on elligator generally
16:10:57 <shelikhoo> dcf1: We need to first understand the exact nature of this connection failure bug, and try upgrading the both the server and the client then observe consequence is one of the way for us to understand this.
16:11:28 <dcf1> Also https://www.reddit.com/r/crypto/comments/fd9t3m/elligator_with_x25519_is_broken_what_workaround/, almost 2 years old now
16:11:30 <shelikhoo> If it does not go away with updating both the client and the server, then this issue is not related to server
16:12:01 <dcf1> arma2: thanks, that would be helpful. Okay, a factor of 8 is in line with my intuition.
16:12:24 <arma2> (2 factors of 8 = factor of 64, to be clear)
16:12:35 <shelikhoo> It might be just a bug introduced in client, than handshake version compatibility issue
16:13:07 <meskio> I agree with you shelikhoo we need to test that
16:13:12 <arma2> dcf1: ok i will mail you and aaron. and cc shell and meskio
16:13:21 <meskio> thanks
16:13:48 <arma2> and yes, understanding the failing-to-connect bug sounds super useful. we had a user in #tor yesterday who was experiencing it (in retrospect) and we just told him to keep trying
16:13:49 <meskio> I guess here we should first decide if this is a reason to roll back to a previous version in TB while we try to fix it
16:13:57 <dcf1> https://gitlab.com/yawning/obfs4/-/blob/77af0cba934d73c4baeb709560bcfc9a9fbc661c/internal/README.md
16:14:08 <dcf1> "Representatives created by this implementation will correctly be decoded by existing implementations." "As the representative to public key transform should be identical, this change is fully-backward compatible (though the non-upgraded side of the connection will still be trivially distinguishable from random)."
16:14:48 <dcf1> So the author says it's backward compatible; maybe that's not actually true for some mathematical reason, or there could be an implementation bug.
16:15:41 <dcf1> I'm thinking a little bit about the potential added distinguishability of a client making multiple repeated connections to the same bridge (not really the overriding concern when using default bridges though)
16:16:39 <arma2> does the tor browser obs4proxy auto retry these connections? or if there is one bridge does it just sit there
16:16:57 <arma2> (until you try a new application level request, and then it marks the bridge up and tries again, once)
16:17:17 <dcf1> I'm not sure if it is tor or obfs4proxy that is retrying the same bridge, but something is.
16:18:44 <cohosh> i'm impressed with the test suite tails has that caught this issue: https://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/40804#note_2778826
16:19:35 <arma2> is it correct to summarize as "an upgraded client can only sometimes connect to a non-upgraded bridge"?
16:20:12 <arma2> seems like we want to fill out the 2x2 matrix of upgraded/non-upgraded client/bridge
16:20:13 <dcf1> arma2: that sounds right -- with the caveat that we haven't tested a connection to an upgraded bridge
16:20:59 <dcf1> so arma2 is going to connect us with aaron johnson to help understand what is going on with the mathematical transform and whether that could be causing connection failures
16:21:17 <dcf1> meskio is wondering if TB should downgrade obfs4proxy until we figure it out, about that I don't really have an opinion
16:21:40 <dcf1> It's a few users whom the log messages bother in the issue tracker and the forum, it seems
16:22:03 <meskio> I guess the visible thing for most users is taking a bit longer to connect
16:22:14 <dcf1> If an upgraded -> upgraded connection works, then additionally we can make plans for getting server operator to upgrade (which includes e.g. getting an upgraded version in Debian)
16:22:39 <meskio> the upgraded version is already in debian
16:22:52 <meskio> but haven't poked people about it, or updated the docker image
16:23:01 <arma2> which debian is it?
16:23:09 <meskio> sid
16:23:12 <dcf1> Possible idea: if a non-upgraded server is detectable from the client (or detectable some fraction of the time), we can log a message to that effect, which can lead users to choose a different bridge or get their bridge to upgrade (if it's their own privately operated bridge, ofr example)
16:23:15 <meskio> we should work on getting it in bakcports
16:23:24 <meskio> but I think we need to wait until is in testing
16:24:30 <arma2> dcf1: yes, it is my understanding that 7/8 of the time, the client can tell, and could pass a message up the chain or something
16:25:31 <dcf1> great, that too is in line with my expectations. The client can do whatever distinguishability attack the censor would do.
16:26:03 <dcf1> that's all I had to say on this topic, I will transcribe the high points into the meeting notes
16:26:15 <meskio> thanks
16:26:34 <arma2> cohosh: do you remember if it was really aaron johnson who told us this? i am having trouble confirming my distant memory
16:27:01 <meskio> I think it makes sense not to rollback the version for now, and see if we can fix it, or upgrade people, soonish
16:27:01 <cohosh> arma2: yep
16:27:23 <arma2> yes you remember? and yes it was? and yes i am having trouble? :)
16:27:30 <cohosh> it was aaron
16:27:31 <shelikhoo> Yes, maybe understand what is happening first....
16:27:41 <arma2> awesome, thanks
16:27:46 <shelikhoo> the next topic is about the utls support we are going to have at Snowflake client for contacting broker
16:27:59 <cohosh> i thnk it's worth looking at our process, maybe a bit later to see how we can detect breaking changes earlier
16:28:06 <shelikhoo> should we enable it by default?
16:28:18 <meskio> I can do some tests and try to reproduce it, and see if upgrading would solve the problem
16:28:19 <cohosh> (like how tails did it)
16:28:37 <meskio> shelikhoo: what are the drawbacks of enabling it?
16:28:37 <arma2> cohosh: agreed. maybe we can even team up with tails, rather than reconstructing what they have
16:29:20 <shelikhoo> the utls implemented some old browser's fingerprint, so for attackers aware of it, it will standout
16:29:39 <meskio> stand out more than go TLS?
16:30:14 <cohosh> shelikhoo: in doing this work, did you get of sense of whether utls is well maintained?
16:30:35 <cohosh> i guess no if the available fingerprints are old?
16:30:47 <shelikhoo> I can't say how do they compare, but there are use of Go language other than anti-censorship
16:31:01 <meskio> true
16:31:02 <shelikhoo> but utls is exclusively used for anti-censorship
16:31:59 <shelikhoo> No, the maintainers is not adding fingerprints of more recent browsers in a timely manner
16:32:44 <meskio> do you have any idea how hard will be to produce a new updated fingerprint?
16:33:24 <shelikhoo> Chrome 83 is the last version in uTLS, but the main stream Chrome is at 97
16:34:09 <shelikhoo> I didn't investigate the work required for adding new fingerprints
16:34:54 <dcf1> I too was disappointed recently looking at the state of fingerprints in uTLS.
16:35:00 <shelikhoo> Based on common sense, adding new extensions to Client Hello isn't hard. The hard thing is supporting some extensions that Go do not support
16:35:21 <dcf1> shelikhoo: One suggestion is to get in touch with Psiphon; I believe they are users of uTLS and perhaps have some insight or internal tweaks.
16:36:04 <shelikhoo> dcf1: Yes, I will do that later.
16:37:41 <shelikhoo> The way TLS fingerprint issue is solve in V2 is by browser forwarding
16:38:21 <shelikhoo> it is a bring your own browser model, so users just open a URL, and the webpage will connect to both V2 and remote server
16:38:26 <dcf1> this is what we used to do with meek, before switching to obfs4proxy for meek in tor browser
16:38:38 <shelikhoo> then relay traffic
16:38:50 <dcf1> except that we used the version of firefox bundled with tor browser, not bring your own browser
16:38:57 <meskio> TB is already a browser, would be nice if we could use it
16:39:14 <dcf1> meskio: well, it actually some with a lot of practical problems
16:39:16 <meskio> ahh, so that was done before...
16:39:38 <meskio> is the fingerprint of chrome the same for desktop and android?
16:39:38 <dcf1> one of which is that the tor browser Firefox ESR is generally older than the general population of Firefox users
16:39:58 <meskio> there are so many old androids with old versions of chrome around, I don't think they can block by that fingerprint
16:40:05 <dcf1> shelikhoo: https://github.com/rod-hynes is the Psiphon developer I know has worked on/with uTLS
16:41:04 <shelikhoo> dcf1: Yes. I will try this contact this person if necessary
16:41:28 <meskio> good, should we wait to make a decision about uTLS to see what psiphon people thinks about it?
16:41:42 <dcf1> shelikhoo: This is the ticket where meek-client gained the ability to use uTLS, in addition to browser forwarding: https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/meek/-/issues/29077
16:42:16 <dcf1> And here is where the decision was made to use obfs4proxy with uTLS and its own meek implementation, for the same of having one fewer binary to ship: https://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/29430
16:43:09 <dcf1> (Aside, if obfs4proxy-0.0.13's removal of uTLS support causes problems for the meek implementation in Tor Browser, there is always the option of going back to the mainline meek-client implementation. It still works, as far as I know.)
16:44:35 <shelikhoo> dcf1: Yes. I will have a look at these tickets to get the context.
16:44:41 <keroserene> hi all/4
16:44:46 <dcf1> That's why obfs4proxy's implementation is called "meek_lite": originally it lacked the browser camouflage feature, so it was considered lighter but more distinguishable. It became less "lite" when it gained uTLS capability.
16:44:52 <dcf1> Oh wow hi Serene!
16:45:15 <meskio> keroserene: hello, nice to see you here
16:45:21 <keroserene> been a minute :) here to help if I can
16:45:23 <cohosh> :D
16:45:25 <shelikhoo> Hi~ keroserene !
16:46:01 <meskio> keroserene: not sure if you got the context, but we are talking about snowflake connecting to the broker using uTLS
16:46:46 <keroserene> I've been here since the start of today's mtg getting context, indeed, awesome progress since last...
16:46:56 <meskio> :)
16:47:54 <meskio> dcf1: about obfs4proxy, I think we should bring back uTLS into meek_lite
16:48:13 <meskio> or I used to think it, now shelikhoo is bringing some good questions to it
16:49:02 <irl> i've asked acute to pause on uploading the obfs4proxy backport package so that we don't get a ton of users upgrading automatically to a broken thing
16:49:39 <arma2> keroserene: welcome! 'indistiguishable from background while quiet' indeed :)
16:49:55 <meskio> BTW ooni has seeing in the past go TLS being blocked by fingrprinting (don't recall the country)
16:50:08 <shelikhoo> I think it is Iran
16:50:42 <shelikhoo> I have received some report about this. China is also doing this to a lesser extent
16:50:59 <meskio> irl: thanks, let's investigate before doing that move
16:51:35 <meskio> it looks like is a topic we should continue talking next week
16:51:39 <meskio> anything more about it?
16:51:44 <meskio> should we move to the reading group?
16:52:31 <meskio> this week we are discussing "Weaponizing Middleboxes for TCP Reflected Amplification"
16:52:39 <meskio> https://censorbib.nymity.ch/#Bock2021b
16:53:15 <meskio> I can try to give a summary
16:53:55 <dcf1> https://geneva.cs.umd.edu/weaponizing
16:54:00 <meskio> the paper proposes mechanisms to use middleboxes, like censorship ones, to amplify DoS attacks
16:54:10 <dcf1> I'm afraid I have to split after about 5 minutes
16:54:42 <meskio> they do use genetic algorithms to find out what kind of packets produce higher amplification
16:55:01 <meskio> to produce attacks of up to  50,000x
16:55:29 <meskio> dcf1: that is a pity, if you want we can move the conversation to next week
16:56:04 <dcf1> the great thing to me is that the amplification attacks use TCP, which was thought to be impossible, because an off-path attacker needs to guess sequence numbers and prevent the victim from terminating the attack with a RST
16:56:35 <meskio> yep, this is amazing
16:56:43 <dcf1> but because middleboxes are designed to be lax about understanding protocols (because they e.g. may only see on direction of a conversation), you don't need to strictly conform to the protocol to get them to send data
16:57:20 <dcf1> meskio: it's okay, I read the paper and I can read the discussion backlog
16:57:25 <meskio> :)
16:57:51 <meskio> it was very interesting how they found so many router loops, so they could trigger the middlebox response many times with the same package
16:58:28 <dcf1> so like with the GFW injecting 3 RSTs for 1 SYN, you get a small amplification factor (but greater than 1.0), but with censor boxes that inject whole blockpages you can get a large factor
16:59:20 <dcf1> Also nice to see the cross-project use of resources, they used Quack reports from Censored Planet in the intial pass to find likely vulnerable middleboxes and the URLs that would trigger them
17:00:01 <meskio> they also reuse their own genetic algorithm, used before for censorship evasion
17:01:24 <meskio> something I found weird in the paper is that they are suprised to find so many middleboxes outside the 'standard' censoring countries
17:01:33 <meskio> they even talk about non-censored countries like the US
17:01:37 <meskio> I don't know much about the US
17:01:44 <meskio> but in europe middleboxes are common
17:01:58 <meskio> and countries do censor a lot of websites in internet
17:02:12 <meskio> (about piracy, gambling, ...)
17:02:58 <meskio> anywa, I had a lot of fun reading the paper, is cool to see the censors being trolled
17:05:00 <shelikhoo> One of the issue that is not directly related to this paper but maskio's observation is that there are censorship middle box in region with less internet censorship. Who developed these middle boxes. If these boxes are from somewhere like China, it is possible for these box to send traffic data back to China to assist censorship at home?
17:05:47 <meskio> in spain one of the mayor ISPs uses fortinet middleboxes
17:05:53 <shelikhoo> Let's say a Tor bridge connect to Tor network in standard Tor protocol, and accept anti-censorship traffic
17:06:06 <meskio> https://www.fortinet.com/
17:06:44 <irl> middleboxes are everywhere
17:06:47 <meskio> wich is US based
17:06:50 <shelikhoo> it is possible for adversaries to know about its traffic to Tor network, and then make restriction to its  anti-censorship traffic
17:07:09 <irl> they broke the internet so much that we invented quic
17:08:12 <irl> middleboxes are often used for traffic engineering purposes so they appear anywhere you have traffic at scale
17:08:26 <irl> they also tend to be very buggy
17:08:26 <meskio> but yes, there is all this paranoia if chinese hardware sends data to china, who knows, but it looks like this paranoia is also comming from intered parties in the US
17:08:46 <meskio> s/intered/interested/
17:09:33 <shelikhoo> although my interest is remove these middlebox, not replace them with US one
17:09:48 <meskio> I agree
17:09:59 <meskio> but I see more and more ISPs around deploying them
17:10:08 <meskio> and governments asking them to do so
17:10:27 <meskio> in spain most ISPs do block content by DNS, no middleboxes yet, but they are slowly catching up
17:10:48 <shelikhoo> Yes....
17:12:39 <meskio> this one is a couple of years old, but describes how every ISP in spain blocks a pro-abortion website: https://sindominio.net/sincensura/en/post/informe/
17:12:45 <meskio> (and abortion is legal...)
17:14:04 <shelikhoo> Yes... I think this meeting is about 15 minutes over the schedule...
17:14:15 <meskio> yes, let's finish here
17:14:24 <shelikhoo> Yes. Thanks~
17:14:31 <meskio> #endmeeting