16:01:54 <asn> #startmeeting
16:01:54 <MeetBot> Meeting started Fri Jan  2 16:01:54 2015 UTC.  The chair is asn. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:01:54 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
16:02:05 <asn> so, let's start with some status reports.
16:02:13 <ohmygodel> i can start
16:02:13 <asn> so over the past 1.5 week,
16:02:16 <asn> ah
16:02:17 <asn> go for it, ohmygodel .
16:02:22 <ohmygodel> ah ok thx
16:02:40 <ohmygodel> i added stuff to the tech report
16:02:44 <ohmygodel> i sent it emails with patches to all of you
16:02:51 <asn> yes i saw that email. haven't had time to read it yet :(
16:02:56 <karsten> received, not applied yet.
16:03:04 <asn> could you briefly describe what you changed?
16:03:12 <karsten> ohmygodel: are you okay with having your name in the git commit?
16:03:14 <ohmygodel> i did in the emails
16:03:19 <asn> ohmygodel: ok then.
16:03:21 <ohmygodel> yes karsten i am fine with that
16:03:21 * asn reads mail
16:03:25 <karsten> ohmygodel: ok.
16:03:39 <ohmygodel> that is all i did for HS stats
16:03:41 <ohmygodel> end report
16:03:45 <asn> thank you
16:03:59 <asn> so over the past week,
16:04:04 <asn> I mainly did CCC.
16:04:11 <asn> which was very interesting.
16:04:20 <asn> during CCC i did an impromptu meeting with karsten
16:04:24 <asn> where we looked at some data
16:04:37 <asn> and kept some notes about how to present the data etc.
16:04:50 <asn> we kind of decided which graphs we should shiow during the jan12 meeting
16:05:00 <asn> and we kind of decided on how to do extrapolation.
16:05:22 <asn> I also spent a bit of time talking with Gareth about his talk
16:05:28 <asn> (the final Tor talk of CCC)
16:05:34 <asn> and interpretering his results.
16:05:50 <asn> interestingly, our SponsorR statistisc were helpful in interprtering his results.
16:06:07 <asn> especially the "HS traffic is about 1.5% of total network traffic" statistic
16:06:16 <asn> which becamse helpful faster than we expected it to be.
16:06:30 <dgoulet> awesome
16:06:32 <ohmygodel> yay stats win !
16:06:37 <asn> so that's what I did about SponsorR laetly.
16:06:41 <asn> next please.\
16:06:49 <robgjansen> can we queue your extrapolation plan for later discussion?
16:06:56 <karsten> yes, please.
16:06:57 <robgjansen> i’d like to hear what you came up with
16:07:00 <karsten> I'd like to talk about that.
16:07:02 <asn> yes
16:07:09 * robgjansen starts
16:07:13 <asn> robgjansen: go!
16:07:27 <robgjansen> i’ve been working with dgoulet on running benchmarks with shadow
16:07:37 <robgjansen> i set up an ec2 node that we are sharing
16:07:53 <robgjansen> it runs shadow and tor linked to lttng
16:08:17 <robgjansen> we have been debugging issues related to lttng tracepoints not showing up correctly
16:08:24 <robgjansen> made some mods to shadow to support that
16:08:32 <robgjansen> still working out bugs, so its not funcitonal yet
16:08:36 * robgjansen ends report
16:08:50 <asn> thanks!
16:08:59 <asn> what kind of bugs are there?
16:09:00 <asn> lttng bugs?
16:09:04 <asn> tor bugs? or shadow bugs?
16:09:06 <asn> or combo bugs?
16:09:32 <robgjansen> ahh, well it has to do with the way that shadow hijacks functions like socket, fnctl, etc
16:09:42 <dgoulet> issues on making them work together, shadow and lttng userspace are intense inprocess library that hijacks symbols
16:09:54 <asn> i see
16:10:01 <asn> and they are fighting with each other?
16:10:13 <robgjansen> yes shadow is hijacking lttng
16:10:21 <robgjansen> :)
16:10:25 <asn> ack
16:10:27 <asn> ok who next?
16:10:33 * dgoulet can go
16:10:36 <asn> dgoulet: please
16:11:23 <dgoulet> very fast, mostly working with robgjansen here on the above ^ ... was afk for the holidays a bit so very few for SponsorR except shadow work, next week will be the one I wrap up HS performance results for Jan 12th
16:11:38 <dgoulet> and after that hopefully, analyse and improve perforamnce
16:11:41 * dgoulet done
16:11:46 <asn> great.
16:12:11 * karsten can go next.
16:12:14 <asn> please
16:12:19 <karsten> - Finished an early analysis of hidserv-stats and posted it to #13192.
16:12:19 <karsten> - Reviewed teor's patch to #13192.
16:12:19 <karsten> - Did another analysis and just uploaded it here: https://people.torproject.org/~karsten/volatile/hidserv-stats-analysis.pdf
16:12:22 <karsten> - Talked more to Donncha about detecting malicous HSDirs.
16:12:26 <karsten> end of report.
16:12:31 <asn> nice and tight
16:12:33 <asn> ok
16:12:42 <asn> so we are done with status reports.
16:12:52 <asn> let's move to discussion phase.
16:12:57 <asn> what do we want to talk about?
16:13:02 <karsten> extrapolation.
16:13:03 <asn> we want to talk about extrapolation?
16:13:11 <asn> we want to talk abut what we should be doing next week
16:13:17 <karsten> also, should we ask operators of fast relays to enable our stats?
16:13:25 <karsten> yes, next week.
16:13:30 <asn> karsten: how many stats are we receiving now?
16:13:33 <ohmygodel> id like to discuss new measurement frameworks
16:13:41 <karsten> hrm, that was too short. yes, we should talk about what we should do next week.
16:14:08 <karsten> asn: now, or when we discuss extrapolation of stats?
16:14:15 <asn> OK, let's start with extrapolation. that many people find exciting.
16:14:26 <asn> karsten: would you like to do an intro?
16:14:26 <karsten> sure. /me looks.
16:14:34 <karsten> https://people.torproject.org/~karsten/volatile/hidserv-stats-analysis.pdf
16:14:45 <karsten> that's from yesterday or so.
16:15:10 <karsten> we have 418 stats reported by relays since dec 21.
16:15:11 <asn> interesting.
16:15:13 <ohmygodel> where is the data coming from
16:15:16 <karsten> not including very recent ones.
16:15:20 <asn> karsten: that's good.
16:15:28 <karsten> data comes from extra-info descriptors and consensuses.
16:15:28 <ohmygodel> so this is people with HiddenServiceStats enabled ?
16:15:31 <karsten> yes.
16:15:35 <asn> karsten: i think we are ok as far as stats volume goes?
16:15:38 <ohmygodel> ok
16:15:41 <karsten> asn: depends.
16:15:50 <karsten> look at pages 3 and 7.
16:15:54 <asn> ok
16:16:15 <karsten> page 7 looks okay. lots of hsdirs, all with similar fractions observed.
16:16:22 <asn> what is the cut line?
16:16:25 <karsten> but page 3 says that we don't have enough fast relays reporting stats.
16:16:40 <karsten> an arbitrary threshold I picked for drawing the graph on the next page.
16:16:58 <karsten> which is too low for page 3.
16:17:29 <karsten> let me paste the description of these pages somewhere.
16:18:00 <karsten> http://pastebin.com/2Xpv4J6C
16:18:04 <asn> yes, I can imagine that RP-related stats would benefit from some faster relays
16:18:11 <asn> since only fast relays will give big numbers there.
16:18:15 <karsten> right.
16:18:35 <karsten> basically, I tried two different extrapolation methods.
16:18:43 <karsten> (neither of which being backed by real maths...)
16:18:46 <asn> omg such cloudflare
16:19:10 <robgjansen> i would suggest using 3.5% on page 7 instead of 2%
16:19:11 <asn> ok got it.
16:19:17 <karsten> pages 1 and 2 and 5 and 6 are extrapolations based on reports by single relays.
16:19:28 <karsten> robgjansen: plausible.
16:19:49 <robgjansen> which i think would bring down the results on page 8
16:19:53 <asn> 5 is extrapolated?
16:20:12 <asn> isn't it the actual measurements of single relays?
16:20:14 <karsten> 5 is reported values.
16:20:55 <karsten> to continue the sentence above, 3 and 4 and 7 and 8 add up all reports and extrapolate from there.
16:21:37 <karsten> robgjansen: I'll raise the threshold. the hope is to have > 3.5% on dec 31 and jan 1 anyway.
16:21:45 <karsten> it just takes some time for reports to come in.
16:21:54 <robgjansen> ok, great
16:22:12 <robgjansen> would it make sense to separate into 2 pdfs? reported and estimated/extrapolated?
16:22:31 <robgjansen> nah, nevermind
16:22:34 <asn> so 7 is calculating the fraction of total onion addresses we see? but total oniona dddresses in an extrapolated number, right?
16:22:38 <karsten> sure. this is just what ggplot produces. a pdf with all results.
16:23:31 <karsten> 7 is the sum of all fractions of relays reporting stats.
16:23:40 <karsten> 8 is extrapolated.
16:23:42 <ohmygodel> karsten id be interested in the formulas used to do the extrapolations
16:23:53 <ohmygodel> e.g. how you calculated the per-relay fractions to adjut by
16:24:05 <karsten> reported number / fraction of observed values = estimated number in the network.
16:24:07 <ohmygodel> *adjust
16:24:27 <karsten> ah, fractions come from consensuses.
16:24:38 <karsten> for traffic it's simply consensus weight fraction.
16:24:55 <ohmygodel> yes that latter number
16:25:10 <karsten> for .onions it's fraction of identifier space that the hsdir was responsible for.
16:25:13 <ohmygodel> im interested in things such as how you are filtering by flag and applying position weights
16:25:35 <karsten> nothing, just consensus weight fraction. happy to tweak that.
16:26:13 <ohmygodel> ic, because i think you will essentially never be selected as an RP if you don’t have the Running and Fast flags
16:26:26 <karsten> ah ok.
16:26:32 <karsten> I didn't look that up, to be honest.
16:26:33 <ohmygodel> also guards and exits would obviously have smaller probs as RPs than their consensus weights would otherwise imply
16:26:45 <karsten> Running is not an issue, but I'm not considering Fast. I can fix that.
16:26:45 <asn> need_uptime = 1
16:26:45 <asn> need_capacity = 1
16:26:45 <asn> need_guard = 0
16:26:45 <asn> allow_invalid = 1
16:26:45 <asn> weight_for_exit = 0
16:26:47 <asn> need_desc = 1
16:26:50 <asn> i think these are the flags of RPs
16:26:55 <asn> it needs to be stable and fast.
16:27:06 <ohmygodel> Stable! Interesting.
16:27:21 <asn> i think so
16:27:23 <asn> let me find the code
16:27:34 <robgjansen> ohmygodel: how much will the positional probs change the consensus weights?
16:27:35 <ohmygodel> That makes a non-trivial difference.
16:28:03 <robgjansen> ohmygodel: ie how much percent roughly is non-trivial
16:28:08 <ohmygodel> A lot for exits
16:28:25 <karsten> so, which weights do I use here?
16:28:53 <ohmygodel> probably > 10% reduction in available relays
16:29:17 <robgjansen> ohmygodel: k thanks
16:29:27 <ohmygodel> I believe that the RP is selected using Middle weights
16:29:29 <karsten> Wmx?
16:30:18 <karsten> Wmg - Weight for Guard-flagged nodes in the middle Position
16:30:24 <karsten> looks like it.
16:30:44 <karsten> okay, I can throw out relays without Stable and without Fast flag, and apply Wmx weights.
16:30:50 <asn> fwiw, I think RPs are chosen in connection_ap_handshake_attach_circuit():
16:31:45 <asn> where it dose:
16:31:46 <asn> retval = circuit_get_open_circ_or_launch(
16:31:46 <asn> conn, CIRCUIT_PURPOSE_C_REND_JOINED, &rendcirc);
16:32:08 <karsten> okay.
16:32:32 <karsten> I'll also read code after the meeting and see if I should do more than what I said above.
16:33:07 <ohmygodel> yes
16:33:07 <ohmygodel> That should be double-checked, though. I have never paid especially close attention to HS path creation, and it’s not explained well in rend-spec.txt.
16:33:07 <ohmygodel> No I meant that being selected as a middle should be double-checked
16:33:07 <ohmygodel> Excellent. Also, can we see the code you are using?
16:33:07 <ohmygodel> Then I can later ask questions like: “What factor is Karsten using to adjust HS publishes?”
16:33:07 <ohmygodel> (I assume the answer to that question is 6/num_hsdirs for all HSDirs)
16:33:38 <asn> :)
16:33:45 <ohmygodel> cool. if you wrote anything up about what you found, i would read it
16:33:45 <karsten> ohmygodel: let me pastebin the code, too.
16:33:52 <asn> you are also using the number 6.
16:33:59 <karsten> I didn't clean it up yet though. but hey.
16:34:07 <ohmygodel> it would be good to consider adding details to rend-spec.txt also
16:34:11 <asn> ohmygodel: yes.
16:34:30 <asn> ohmygodel: that's how I had those flags ready. i have some notes that I've been planning to incorporate to rend-spec.txt but time--.
16:34:53 <karsten> ohmygodel: http://pastebin.com/cyEEyav1
16:35:15 <ohmygodel> asn: what did you mean by “that's how I had those flags ready” ?
16:35:30 <asn> i meant the ones I pasted before. need_uptime = 1, etc.
16:35:42 <karsten> regarding the number 6, ...
16:35:49 <ohmygodel> oh i understand. cool!
16:36:15 <karsten> I did something slightly different:
16:36:21 <asn> ohmygodel: HS path selection should be better specified both in rend-spec.txt and in rend-spec-ng.txt
16:36:31 <karsten> I looked at the whole range that an HSDir is responsible for.
16:36:34 * asn listens to karsten
16:36:45 <karsten> from 3 HSDirs earlier in the ring until the own fingerprint.
16:36:58 <karsten> (/me operates from memory there and hopes that is even correct..)
16:37:22 <karsten> then I introduced the magic number 4:
16:37:37 <karsten> each service publishes two replicas of its descriptor to the ring.
16:37:55 <karsten> and it changes ring positions once every 24 hours, which almost never aligns with a relay's stats interval.
16:38:05 <karsten> that's 2 x 2 changes to see a service over the day.
16:38:10 <karsten> hence 4.
16:38:45 <karsten> not sure if this is a useful discussion on irc. asn spent a tea or two talking about this...
16:39:10 <ohmygodel> ok yeah it makes sense though thanks
16:39:22 <asn> as I understand it, karsten uses the slice of 3 HSDirs as his unit
16:39:26 <asn> not a single HSDir.
16:39:29 <karsten> yep
16:39:43 <asn> and also, he takes in account that HSes will rotate once during his statistics interval
16:39:57 <asn> so he expects the same HS to register itself to two of our HSDirs during a singkle statistics interval
16:40:07 <ohmygodel> although you need to be careful when you add up all the .onions seen
16:40:12 <asn> that's why he doubles up his number.
16:41:02 <ohmygodel> i mean when you add them up *across* relays
16:41:02 <ohmygodel> to come up with a more robust total estimate than just based on those observed at a single relay
16:41:02 <karsten> well, it may be that we're not handling overlapping sets correctly.
16:41:18 <ohmygodel> ok
16:42:05 <karsten> I started describing the method in a tech report format. would you want to read/review that?
16:42:23 <ohmygodel> im still dont totally understand your inference calculation, but yeah, maybe irc isnt the place
16:42:27 <ohmygodel> yes i would definitely want to read that
16:42:46 <asn> it might also be worth trying the alternative extrapolation technqiue that ohmygodel started describing
16:42:52 <asn> and then seeing how much the resulst differ.
16:43:09 <karsten> rather than reading 500+ lines of java plus those R lines I didn't even paste yet. okay, great, will send that to you. (damn lag)
16:43:24 <asn> our results give us approx 30k to 35k HSes.
16:43:36 <asn> Gareth's results gave him 45k HSes 6 months ago. who knows who is correct.
16:43:47 <karsten> which alternative extrapolation technique?
16:43:48 <asn> we are not too far away though.
16:43:51 <ohmygodel> yeah i do prefer english to java haha
16:44:20 <ohmygodel> well also our results include added noise
16:44:20 <ohmygodel> correct ?
16:44:23 <asn> the one where you consider HSDirs as individual units, instead of considering their whole slice.
16:44:27 <asn> ohmygodel: yes.
16:44:43 <karsten> ah hmm. isn't that less precise?
16:44:57 <karsten> because we know what fraction of descriptors we're responsible for.
16:45:14 <ohmygodel> yeah i agree your method is better
16:45:16 <karsten> feels like throwing away that information.
16:45:22 <asn> that's true.
16:45:48 <karsten> ok. then let me write this down in english rather than java. :)
16:46:08 <ohmygodel> btw are you just taking bin_center+noise as the true value
16:46:13 <ohmygodel> even for negative values, for example ?
16:46:18 <karsten> almost
16:46:24 <karsten> I'm subtracting bin_size / 2.
16:46:26 <asn> which bin_center? :)
16:46:29 <karsten> oh
16:46:32 <asn> i guess the closest.
16:46:41 <asn> since that's more likely to have been returned by the laplace distr.
16:46:46 <karsten> yes, bin_center is what I'm using.
16:46:48 <asn> (now that we are applying additive noise second)
16:47:09 <karsten> hang on. confusion.
16:47:30 <karsten> what we do is: we measure a value, we round up to the next multiple of something, we add laplace noise.
16:47:44 <karsten> what I did in the analysis: subtract bin_size / 2 from whatever was reported.
16:47:55 <asn> why not add? :)
16:47:55 <ohmygodel> ok yes that muliple was what i was calling bin_center
16:47:58 <asn> that's a design decision I guess.
16:48:13 <karsten> because we added 0..bin_size-1 before.
16:48:14 <ohmygodel> yeah i dont agree with that correction
16:48:42 <karsten> why not?
16:48:43 <ohmygodel> wait you round up ?
16:48:43 <ohmygodel> why dont you round to the nearest ?
16:48:54 <ohmygodel> because it should be more accurate to round to the nearest
16:48:56 <asn> we can't really reverse the binning.
16:49:05 <asn> we should try to reverse the laplace noise.
16:49:06 <karsten> ah, in the tor code we round up.
16:49:14 <ohmygodel> well at this point no.
16:49:15 <ohmygodel> karsten: yeah in the tor code
16:49:29 <ohmygodel> ok just a suggestion: round to the nearest in the tor code
16:49:35 <karsten> ugh
16:49:42 <karsten> we can't really change that at this point.
16:49:45 <ohmygodel> also, for post-noise correction
16:49:48 <ohmygodel> ok no big deal
16:50:01 <ohmygodel> really not at all a big  deal
16:50:08 <karsten> ok
16:50:19 <ohmygodel> then i do agree with your correction
16:50:24 <karsten> phew :)
16:50:31 <karsten> I mean, happy to change that.
16:50:33 <karsten> that's easy.
16:50:34 <ohmygodel> actually it would seem to make more sense
16:50:35 <asn> as I see it,
16:50:40 <karsten> just changing the tor part is hard.
16:50:44 <asn> the correct thing to do with such a value (to bring it closer to the original value)
16:50:54 <ohmygodel> to just round to the halfway between two muliples
16:50:54 <asn> is to round up or down the value to the nearest bin_center.
16:51:35 <karsten> asn: you mean after subtracting bin_size/2?
16:51:37 <ohmygodel> because you take the current bin as the most likely one, and then try to minimize expected error
16:51:38 <asn> since laplace is an exponential distr, the nearest bin_center is the most likely original value.
16:51:55 <ohmygodel> yes, asn are saying the same thing
16:52:00 <asn> yes
16:52:23 <karsten> so, subtract bin_size / 2, then round to nearest bin_center?
16:52:29 <asn> karsten: no. i don't see subtracting bin_size/2 in my plan.
16:52:31 <asn> but i might be wrong.
16:52:34 <ohmygodel> no, just round to the center of the current bin
16:52:38 <asn> ye
16:52:39 <ohmygodel> e.g., if your multiple value is 8
16:52:48 <ohmygodel> and youre reported value is 643
16:53:12 <ohmygodel> then you are between multiples 640 and 648
16:53:19 <ohmygodel> so take that as the bin of the true value
16:53:26 <asn> that's also how I see it.
16:53:33 <ohmygodel> and then guess that its at 644
16:53:45 <asn> yeah. and then you can't get deeper. because the binning has smoothed out information.
16:53:49 <karsten> ok.
16:54:21 <karsten> a tiny part in my brain still wants to subtract something. but the rest makes sense.
16:54:43 <asn> ohmygodel: i was also hoping to add a small aura to the graph
16:54:45 <karsten> rounding to a bin center. will do that.
16:55:03 <asn> ohmygodel: so that if our infered value is 600. it would ahve an aura between 540 to 640, which is the error rate.
16:55:11 <asn> ohmygodel: but I'm not sure if this will be that useful now.
16:55:39 <asn> ohmygodel: since the laplace distr goes from -inf to +inf, the aura would have to be eveyrwhere.
16:56:19 <asn> we could add an aura to only the 90% most likely values of the laplace distr, but that's still very big.
16:56:29 <ohmygodel> asn: not sure where you got those numbers, but the error analysis shuold yield a confidence interval something like that
16:56:52 <asn> ohmygodel: the numbers were arbitrary in the example I just gave. and even so they should have been 560 to 640.
16:57:07 <ohmygodel> ha right
16:57:27 <asn> so that was extrapolation for onion addresses.
16:57:32 <asn> which as I understand it is the hard aprt.
16:57:43 <asn> because it has all this overlapping stuff.
16:58:00 <asn> extrapolating RP statistics is probably easier, if we can get the prob of being an RP for each relay.
16:58:15 <karsten> and if we have fast-enough relays.
16:58:20 <ohmygodel> yeah the only wrinkle is getting the probs right, using the flags and weights correctly, etc.
16:58:36 <ohmygodel> to adjust for the probability of a reporting relay seeing HS traffic
16:58:45 <ohmygodel> but we discussed that
16:58:48 <karsten> yup
16:59:17 <karsten> thanks for all the feedback here! I'll fix these things and send new results and a report thing.
16:59:36 <ohmygodel> btw the program manager of this program (chris white) has a phd in statistics, and so he might appreciate any display of stats competence on our part
16:59:40 <asn> karsten: maybe your report could be a diff to the proposal on the "How to use onion statistics" part?
16:59:55 <asn> karsten: since now that part is a bit dry.
16:59:55 <karsten> ohmygodel: hah, very good to know!
17:00:30 <karsten> a diff?
17:00:37 <asn> OK. So there goes the extrapolation discussion topic. We are not really finished yet though, since this week we shoudl think and work moer on it.
17:00:50 <asn> karsten: i meant a patch to  the proposal
17:01:04 <karsten> hmm. plain text, no latex, ....
17:01:05 <asn> since it has a "how to use the  onion statistics" section that isn ot very useful.
17:01:09 <asn> karsten: ah.
17:01:09 <karsten> yeah
17:01:30 <asn> yeah latex might be more covenient I guess.
17:01:34 <karsten> btw, we need to update the spec.
17:01:38 <karsten> I think so, yes.
17:01:39 <asn> oh well, if you prefer latex just do it in latex and then we can link from the proposal.
17:01:43 <ohmygodel> karsten: how about adding it to the tech report ?
17:01:54 <karsten> asn: sounds good.
17:02:08 <asn> so let's move to next discussion topic for now?
17:02:08 <karsten> ohmygodel: sure, why not. let me write is as new latex file for now, and we can include it later?
17:02:12 <karsten> sure.
17:02:22 <asn> which one is that?
17:02:24 <ohmygodel> karsten: sounds good
17:02:35 <karsten> very quickly:
17:02:49 <karsten> any objections if I ask relay operators of the top-10 relays or so to run 0.2.6.2-alpha?
17:02:52 <karsten> with stats enabled?
17:02:54 <asn> no
17:02:55 <asn> please do.
17:02:58 <asn> thanks
17:03:00 <karsten> ok.
17:03:08 <ohmygodel> i did want to briefly discuss other stats collection methodologies
17:03:14 <asn> please do ohmygodel
17:03:34 <karsten> (and then we only have "what to do next week" on the agenda.)
17:03:37 <asn> yes
17:03:56 <ohmygodel> ok so this topic ended last time
17:03:56 <ohmygodel> with asn recommending that i write up a scheme and send it to tor-dev
17:03:56 <ohmygodel> i still plan to do that
17:03:56 <ohmygodel> one thing that i have been considering
17:04:10 <ohmygodel> is how to gauge “implementation difficulty”
17:04:23 <ohmygodel> which includes both writing software and running services
17:04:39 <ohmygodel> the former is harder for me
17:04:46 <ohmygodel> because im not really a software engineer
17:05:00 <ohmygodel> and its hard for me to gauge which libraries could be used in practice
17:05:11 <ohmygodel> i mean, many many crypto tools have *some* implementation
17:05:13 <ohmygodel> any suggestions ?
17:05:23 <asn> hm
17:05:37 <asn> good question.
17:05:47 <asn> usually, if the implementation involves new crypto primitives
17:06:03 <asn> that are not well-established or triviailly  understood
17:06:10 <asn> the implementation cost rumps up quite a bit.
17:06:25 <asn> because it either involves auditing those libraries, or implementing them ourselves.
17:06:48 <asn> the cost of running services is not as high as writing software, but it's definitely some overhead depending on the number of services.
17:06:49 <ohmygodel> so is there a short list of “vetted” crypto libraries ?
17:06:57 <asn> nooot really :(
17:07:17 <asn> djb stuff that are usable are usually trustworthy maybe.
17:07:23 <karsten> don't underestimate the cost of running services.
17:07:27 <asn> things that are in openssl and used by many people are also useable.
17:07:40 <karsten> take a look at the consensus-health list to see how hard it is to keep current directory authorities happy.
17:07:48 <asn> karsten: yes that's true.
17:08:07 <karsten> not saying that necessity to run new services is a show-stopper.
17:08:13 <karsten> but there's a cost.
17:08:20 <asn> ohmygodel: i would suggest to not worry about implementation difficulty at tghis point.
17:08:25 <asn> ohmygodel: write down your scheme
17:08:34 <karsten> asn: think of your guard script thing.
17:08:37 <asn> ohmygodel: and if you can think of ways that improve implementation difficulty, note them down.
17:08:48 <asn> karsten: yeah that's true.
17:08:50 <karsten> yes, that would also be my suggestion.
17:08:54 <ohmygodel> well its sort of all about implementation difficulty: very secure designs already exist
17:09:21 <ohmygodel> but i understand, ill make some attempt at evaluation, but leave it to others that know more about it
17:09:28 <asn> yes
17:09:48 <ohmygodel> i am hoping to propose to chris
17:09:48 <karsten> take the laplace thing as example.
17:09:51 <asn> maybe note down a few differfent designs or design decisions, so that we can choose between them.
17:09:52 <karsten> pretty trivial, in theory.
17:10:00 <karsten> and we still have a pending patch on the ticket.
17:10:09 <asn> ye
17:10:15 <ohmygodel> that he support tor+nrl implementing a new, more secure, and more efficient stats collection cheme
17:10:46 <karsten> that's a worthwhile thing to do.
17:10:46 <asn> ohmygodel: "interesting"
17:10:47 <ohmygodel> of course, we should figure out if this is in fact a good priority for us in advance though
17:11:25 <asn> that's something worth discussing.
17:11:36 <asn> i'm personally not very excited about implementing this. but I could be persuaded maybe.
17:11:41 <ohmygodel> it wasnt exactly what we had envisioned in the project proposal, but i seems important to move forward on HS stats, among several other things (BW measurement, Tor Metrics in general)
17:12:07 <karsten> agreed.
17:12:17 <karsten> and we have that other sponsor who cares about it.
17:12:25 <asn> i want to speak with roger about these things.
17:12:27 <ohmygodel> yes indeed ! it kills two rids
17:12:33 <ohmygodel> asn: me too
17:12:35 <ohmygodel> *birds
17:12:40 <asn> i'm really hoping we can speak with roger before the meeting on jan12
17:12:43 <karsten> poor birds.
17:13:04 <ohmygodel> i have a meeting with him
17:13:10 <ohmygodel> you should show up
17:13:15 <asn> when is it?
17:13:18 <ohmygodel> 1/9/15
17:13:28 <asn> that's when we have the next sponsorr meeting here.
17:13:30 <ohmygodel> oh wait sorry
17:13:30 <ohmygodel> thats this meeting haha
17:13:34 <karsten> hah
17:13:37 <asn> :)
17:13:41 <asn> hopefully roger will show up here.
17:13:43 <karsten> sounds good :)
17:13:47 <ohmygodel> robgjansen: do you have that mtg in your calendar ?
17:13:53 * robgjansen checking cal
17:13:55 <karsten> speaking of, things to do this week?
17:13:59 <asn> i will send hima n email to remind him.
17:14:13 * karsten already has a long list from this meeting. is everyone else covered?
17:14:13 <asn> ok
17:14:20 <robgjansen> ohmygodel: no, just this HS meeting in tor-dev
17:14:24 <asn> yes you are probably filled karsten
17:14:34 <asn> i would like to help evaluate your stuff
17:14:38 <asn> like read the mail youa re going to send to tor-dev
17:14:42 <asn> and maybe read the java code
17:14:44 <ohmygodel> damnit i definitely recall scheduling something for LIGHTS+MADPATHS in january
17:14:47 <asn> and I should also really curate the tech report.
17:15:07 <karsten> asn: do you want to apply ohmygodel's patches, or should I do that?
17:15:09 <asn> I will try to feel more inspired about this tech report this week.
17:15:10 <asn> i can do it.
17:15:21 <karsten> okay, great.
17:15:35 <asn> btw karsten
17:15:41 <karsten> and sure, any feedback on report and code are much appreciated.
17:15:43 <ohmygodel> my goal will be to write up some possible alternative measurement schemes
17:16:02 <asn> were there any graphs with extrapolation based on all the reports, not on single reports?
17:16:21 <karsten> asn: yes, that's pages 4 and 8.
17:16:38 <karsten> (we didn't discuss those at 31c3, they're new.)
17:16:39 <asn> ack
17:16:43 <asn> ok thx
17:17:06 <asn> ohmygodel: good.
17:17:07 <nicoo> isis: You might want to check the spreadsheet that nickm (
17:17:29 <ohmygodel> hey karsten i actually have changed my mind about post-noise correction
17:17:33 <asn> !
17:17:36 <nicoo> isis: You might want to check the spreadsheet that nickm (?) and Yawning started writing, to sum up the possibilities for non-TLS transport
17:17:57 <ohmygodel> yeah could describe now or send an email
17:17:58 <ohmygodel> you decide asn
17:18:13 <asn> sure describe nao
17:18:22 <ohmygodel> in the example i gave
17:18:34 <ohmygodel> 643 is between multiple 640 and 648
17:18:48 <ohmygodel> but it’s actually most likely to be output if you were rounded to 640 than 648
17:19:07 <ohmygodel> so probably you were a value in (632, 640]
17:19:09 <ohmygodel> that got rounded *up* to 640
17:19:14 <asn> true.
17:19:17 <ohmygodel> that is the most likely case
17:19:18 <ohmygodel> thus to correct
17:19:35 <asn> it's the "nearest bin_center from where you are"
17:19:38 <ohmygodel> you should use 636
17:19:59 <asn> hm.
17:20:00 * asn thinks
17:20:02 <karsten> so, - bin_size/2, round to nearest center?
17:20:26 <ohmygodel> haha i believe so karsten
17:20:29 <ohmygodel> you had good instincts
17:20:35 <karsten> hehe
17:20:56 <ohmygodel> ok sorry back to plans for next week ?
17:20:58 <asn> is the additive noise applied to the bin center or the right side of the bin?
17:21:05 <karsten> right side of the bin
17:21:13 <karsten> we discussed bin center
17:21:17 <karsten> but didn't implement that.
17:21:39 <asn> in that case, I think ohmygodel is correct.
17:21:47 <karsten> ok. cool!
17:21:51 <asn> since 643 is closer to 640, than 648 (which are both right sides of the bin)
17:22:00 <asn> *of bins
17:22:24 <karsten> right. (we could also round to nearest right side of the bin and then go to the bin center.)
17:22:34 <asn> i think now that -bin_size/2 in the beginning might make sense.
17:22:55 <asn> yes. that's more intuitive to me.
17:23:02 <karsten> ok.
17:23:11 <asn> ohmygodel: thanks for catching this.
17:23:19 <asn> karsten: thanks for thinking of this.
17:23:20 <karsten> indeed, thanks!
17:23:27 <asn> ok good.
17:23:34 <asn> so I think we all have things to do this week.
17:23:42 <ohmygodel> yup
17:23:43 <asn> ah, I will be in the same conference as Roger in 5 days!
17:23:50 <asn> that makes it easier to meet with him,.
17:23:56 <karsten> you think that.
17:23:57 <karsten> ;)
17:23:59 <asn> :P
17:24:13 <asn> ok are we all happy?
17:24:18 <asn> fwiw, i will be around irc for the next days
17:24:28 <asn> will be less around irc after the 6th.
17:24:43 <asn> but we can still discuss and adaptively change our plans in the next few days.
17:24:58 <karsten> sounds good. not sure how much I can focus on this on the weekend,
17:25:05 <karsten> but I'll try to get something done soon.
17:25:09 <karsten> next meeting in 1 week?
17:25:11 <asn> yes
17:25:16 <ohmygodel> ok sounds good. ill try to figure out when that LIGHTS/MADPATHS meeting is, assuming it was scheduled.
17:25:18 <asn> that meeting will be our last meeting
17:25:27 <ohmygodel> ok talk to you next week at the latest !
17:25:28 <asn> #endmeeting