16:00:48 <karsten> #startmeeting metrics team
16:00:48 <MeetBot> Meeting started Thu Jan 17 16:00:48 2019 UTC.  The chair is karsten. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:48 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
16:00:51 <karsten> let's start!
16:00:57 <karsten> anything else for the agenda?
16:01:13 <irl> not from me
16:01:54 <karsten> gaba: ?
16:01:55 <gaba> no
16:01:58 <karsten> ok
16:02:02 <karsten> * Onionoo having some slow days (irl?)
16:02:12 <irl> i just wondered if we had an explanation for this
16:02:24 <irl> it fixed itself but it still is worrying that it was falling behind
16:02:40 <karsten> I noticed other issues.
16:02:53 <karsten> [java] Exception in thread "main" org.postgresql.util.PSQLException: FATAL: terminating connection due to administrator command
16:03:00 <karsten> that's from metrics-web.
16:03:19 <karsten> the next run after this one succeeded, but I'm still unsure if we lost any data.
16:03:30 <irl> there have been lots of infrastructure issues recently i think, maybe this is just related to that
16:03:40 <karsten> yes, I think so.
16:03:46 <karsten> the question is just whether it broke something.
16:04:16 <karsten> I don't know yet. it's on my list for tomorrow to check.
16:04:22 <irl> ok cool
16:04:29 <karsten> tomorrow, because it usually takes some time for issues to show.
16:04:37 <irl> i will be away from keys for 2 weeks starting tomorrow, so i can't do any deployments
16:04:44 <irl> but i can help with any reviews that are needed for hotfixes
16:05:04 <karsten> okay!
16:05:23 <karsten> * Funding opportunities (karsten)
16:05:37 <gaba> We had our weekly meeting yesterday and this one was a big topic.
16:05:38 <karsten> gaba: this is still a thing, right?
16:05:45 <gaba> So far we are moving with several proposals.
16:06:10 <gaba> If you have anything then this week would be when to talk about it.
16:06:24 <gaba> but so far Al is working on some proposals ooni and browser related
16:06:43 <karsten> should we do our own proposal, or add something to one of those proposals?
16:06:52 <gaba> So far I think it would be good to include metrics in proposals for other teams too. Al may reach out to any of you about it.
16:07:32 <karsten> that sounds good to me.
16:07:32 <gaba> If you have anything that you think is something that can be a proposal on itself for metrics then that would be good. I can help as a liason for the fundraising team to bring other opportunities here to think about.
16:08:06 <karsten> I think adding something to an existing proposal is easier, given the short period of time left.
16:08:11 <gaba> yes
16:08:16 <irl> +1
16:08:28 <karsten> can you ask al to reach out to us?
16:08:33 <gaba> yes
16:08:44 <karsten> okay, cool!
16:08:56 <karsten> related to this, I had another idea on how to handle ideas lists.
16:09:25 <karsten> for me it's pretty hard to write down ideas, because it feels like committing to something in the future without knowing how that future is going to look like.
16:10:08 <karsten> I could imagine writing ideas if there's another step involved: whenever something might get funded, we then get the chance to flesh out the idea more, possibly write a proof of concept, and possibly say no, if it turns out to be a bad idea.
16:10:37 <karsten> that wouldn't work with two weeks of notice, though.
16:10:52 <gaba> yes, the issue is that we do not get things approved unless we write it
16:10:59 <gaba> and that is the hard part
16:11:03 <gaba> for grant-based projects
16:11:28 <karsten> I was thinking that the proof of concept would have to happen outside of a grant.
16:11:32 <gaba> And we are always sending a proof of concept as the first proposal and then being more specific later.
16:11:52 <gaba> what you mean?
16:12:00 <karsten> ah, I wasn't aware that there are grants for proofs of concept.
16:12:15 <karsten> I was thinking of a week or two to try out if something's a good idea or not.
16:12:34 <karsten> the grants we had in the past always expected things to work out in the end.
16:12:47 <gaba> different opportunities work different. We always send the proposal and we do not know if it will be funded or not. And that is the issue when we do not get funded for something we need to have plan b.
16:12:52 <irl> for science-oriented contracts it is ok to present your findings as "this was a terrible idea"
16:13:10 <karsten> irl: yes, it would be good to have such a model.
16:13:18 <gaba> yes. We can include that in proposals.
16:13:32 <irl> these are generally where our proposal presents a research question as opposed to a methodology
16:13:57 <irl> i think this is something we can add to the hack week agenda
16:13:58 <karsten> however, I'm also thinking about avoiding to ask stupid research questions.
16:14:04 <irl> there are too many options out there
16:14:18 <karsten> yes, I wanted to suggest this, too. talk about this in brussels.
16:14:41 <gaba> we need to find a way to work with the grants team to include metrics proposals or ideas on projects. and +1 for a discussion during hackweek
16:15:15 <karsten> ok.
16:15:43 <karsten> cool. moving on?
16:15:58 <gaba> the issue with metrics is that it touches everything :)
16:15:59 <gaba> yes
16:16:05 <karsten> heh
16:16:10 <karsten> * Clarification on sponsor 19 and discussion on how metrics can be involved (gaba)
16:16:24 <karsten> what should we do and when?
16:16:33 <gaba> yes, I wanted to clarify about this sponsor as it is not a 'traditional' one with clear deliverables
16:16:59 <gaba> In the roadmap that roger is propposing for anti-censorsship there are a few places where metrics can be involved
16:17:14 <gaba> s19 is until may but this roadmap is work to continue after s19
16:17:35 <gaba> so it is something to think about when creating the new roadmap for metrics
16:18:19 <gaba> what part of the work that needs to be done by metrics can be funded by s19 and whatfrom that roadmap is something that the team thinks is a priority
16:18:42 <gaba> comments,questions?
16:19:07 <karsten> we talked about bridgedb stats in mexico.
16:19:25 <karsten> who's working on this on the bridgedb side?
16:19:49 <gaba> right now nobody. I think it will be included in the network team roadmap while in brussels but is not there yet
16:20:15 <karsten> okay. this is not something we can drive on our own. we can help drive it, but this needs a bridgedb dev.
16:20:37 <karsten> but if you say we should consider this for our post-march roadmap, that sounds doable.
16:20:37 <gaba> and is something that the anti-censorship team will be responsible for
16:20:45 <gaba> ok
16:20:47 <gaba> it sounds good
16:21:19 <karsten> if I cc metrics-team on the ticket, will that work?
16:21:25 <gaba> yes
16:22:25 <karsten> okay. added some notes.
16:22:45 <karsten> please remind us if we forget to add this to our next roadmap.
16:22:51 <karsten> despite notes.
16:23:17 <karsten> alright. shall we move on?
16:23:26 <gaba> yes
16:23:31 <karsten> * Moving forward with #28615 (with or without @type for network status entries) (karsten)
16:23:54 <karsten> this seemed like something we might quickly move forward here.
16:24:05 <irl> yes
16:24:18 <karsten> I was thinking that we could add a new @type for detached-signature, but not for the others.
16:24:30 <karsten> we could add other types later.
16:24:31 <irl> from a dev perspective, i would like to have the network-status-entry types
16:24:52 <irl> and strings are cheap
16:24:52 <karsten> ah, hmm.
16:25:19 <karsten> I wonder, would it work to feed in a partial descriptor plus the descriptor type?
16:25:32 <karsten> rather than a partial descriptor plus a partial descriptor identifier?
16:25:50 <karsten> maybe with a flag this-is-a-partial-descriptor-string.
16:26:04 <irl> partial descriptor doesn't have the same meaning as it being a single status
16:26:16 <irl> plus a partial descriptor could also be the headers
16:26:25 <irl> i think network-status-entry keeps it nice and clear
16:26:50 <karsten> it feels like we're starting to clutter things.
16:27:05 <karsten> okay, let's maybe continue on the ticket then.
16:27:08 <irl> ok
16:27:22 <irl> i think we just have a lot of document types to think about
16:27:45 <karsten> well, yes.
16:27:58 <karsten> therefore I'm careful not to add identifiers for non-document types.
16:28:05 <karsten> that is, partial document types.
16:28:20 <karsten> but okay, maybe there are more aspects to consider here.
16:28:22 <karsten> -> ticket
16:28:30 <karsten> * Single psql database with multiple schemas for metrics-web (karsten)
16:28:39 <karsten> (unless there's more on the last topic?)
16:28:46 <irl> we can do it on the ticket
16:28:48 <karsten> ok.
16:29:02 <karsten> this is another thing that might be quick to discuss here.
16:29:13 <karsten> I was thinking that we could have a single psql database for metrics-web.
16:29:25 <karsten> it feels like it could be easier from an operation POV.
16:29:34 <karsten> do you have any prior experience with that?
16:29:56 <karsten> right now I need to open a ticket for creating a database for a new metrics-web module.
16:30:23 <irl> would we expect to merge any of the schemas?
16:30:35 <irl> i.e. share tables between modules?
16:30:36 <karsten> not in the near future.
16:30:49 <karsten> well,
16:31:07 <karsten> what we might do is add tables shared by all schemas/databases.
16:31:17 <karsten> like, a table with descriptor files we already imported.
16:31:25 <karsten> that would be a separate schema then.
16:31:34 <karsten> but, merging tables is not the primary goal.
16:31:44 <karsten> ease of configuration, possibly ease of backup.
16:31:56 <irl> i can't think of any downside
16:32:09 <karsten> okay. good to know.
16:32:26 <karsten> I wasn't sure if was overlooking something obvious. will look closer.
16:32:40 <karsten> thanks! moving on?
16:32:53 <irl> ok
16:32:57 <karsten> * Improving OnionPerf operation (karsten)
16:33:16 <karsten> cf. #29091
16:33:37 <irl> there is work in progress here
16:33:40 <karsten> I don't think we can automate this before the hack week.
16:34:00 <irl> i was talking with hiro about maybe redoing the existing deployments in march/april with the new code
16:34:30 <irl> we use the system tor installed from debian package and also drop the webserver, plus there would be some new nagios tests
16:34:33 <karsten> so, hmm, until then, would you mind taking a quick look every few days whether things are working okay?
16:34:51 <irl> i can look today, but then not for 2 weeks
16:35:29 <karsten> ah, you mean you couldn't fix issues then.
16:35:35 <irl> right
16:35:48 <karsten> (you could probably look at tor metrics and find that something's wrong, but then not do anything.)
16:35:59 <irl> yeah
16:36:00 <karsten> could you ping hiro in such a case?
16:36:12 <irl> i can do that
16:36:25 <karsten> that would be great!
16:37:08 <karsten> thanks! moving on:
16:37:11 <karsten> * Agenda for hack half week (karsten)
16:37:57 <karsten> how about we add topics to the linked pad and talk about those topics next week?
16:38:08 <gaba> I added the one we are drafting for network team there. One topic that would be good for the metrics team to be in is the privcount one.
16:38:18 <karsten> gaba: yes, agreed.
16:38:24 <irl> ok
16:38:26 <gaba> that sounds good
16:39:15 <karsten> what's the K in CKAN?
16:39:21 <irl> knowledge
16:39:25 <irl> it's an open data portal
16:39:37 <irl> to solve the publishing of data and linking to other's data problem
16:39:50 <karsten> I see.
16:40:01 <irl> https://ckan.org/
16:40:07 <irl> (but it can wait, this is not an urgent thing)
16:40:20 <gaba> where do you see the ckan thing? is that in the agenda?
16:40:26 <karsten> on the pad.
16:40:29 <irl> i added it as a possible topic on the pad
16:40:35 <gaba> ahhh, ok
16:40:36 <karsten> the hackweek pad.
16:40:44 <gaba> to publish data from tor?
16:40:56 <gaba> ckan is usually used for open data related to civic tech and government
16:41:01 <irl> tor metrics data and research data from tor
16:41:13 <gaba> ahh, ok
16:41:25 <irl> that could then be hacked up with civic data to see things like how popular is tor in countries that have X level of development
16:41:44 <gaba> yes, it could be interesting
16:42:04 <irl> it is good that you've heard of it
16:42:13 <irl> this means it must be a thing people use
16:42:16 <irl> (or hate)
16:42:20 <gaba> yes, we used it in uruguay in the local gov in montevideo
16:42:27 <gaba> yes, people use it AND hate it
16:42:31 <irl> yay
16:42:35 <irl> maybe it will be fun
16:42:42 <karsten> regarding the pad, maybe add a few words to explain a topic.
16:42:56 <karsten> otherwise these topics can be very broad, or not.
16:43:01 <gaba> one question, the hackweek for metrics will be wednesday, thursday, friday. right?
16:43:20 <karsten> half of wednesday, but yes.
16:43:38 <karsten> I expect to arrive in the early afternoon.
16:43:49 <irl> i arrive on 27th
16:43:51 <karsten> but irl will be around earlier.
16:43:52 <irl> i don't know what day that is
16:43:59 <gaba> ahh, ok
16:44:05 <gaba> 27th is sunday
16:44:30 <gaba> then if we do privcount together with both teams then it makes sense to do it on thursday
16:44:31 <karsten> maybe there are topics that require the network team and _a_ metrics team person. those could possibly happen in the first half of the week.
16:44:33 <gaba> and not wednesday
16:44:44 <karsten> right, privcount should happen in the second half.
16:44:49 <gaba> or we can do privcount only with irl?
16:44:53 <gaba> ahh, ok
16:45:06 <irl> i think it would be good to have both me and karsten for privcount
16:45:28 <karsten> maybe it would be good to know what we're talking about.
16:45:32 <karsten> "privcount" is rather broad.
16:45:43 <karsten> we talked about privcount for several years now.
16:45:52 <gaba> the idea is for teor to give a presentation on what they are doing and then discussion
16:45:54 <irl> for talking to the network team, a timeline for bandwidth lists would be good to discuss
16:45:56 <gaba> ahh, i see
16:46:02 <irl> i could do that without karsten if it helps
16:46:25 <gaba> teor is working on privcount now
16:46:50 <karsten> irl: sounds good to me.
16:47:09 <karsten> just in case we feel we need to talk more, we could possibly schedule a second meeting.
16:47:56 <karsten> okay. let's make more plans during this week and then discuss next week, okay?
16:48:26 <irl> karsten: if we need a second meeting i am planning to be in prague to present the safe measurement guidelines to the ietf in march
16:48:41 <irl> so i can fly to you or prague and then train to the other place and fly home
16:48:53 <karsten> ah, I meant a second session in the second half of the hackweek.
16:48:57 <irl> ah ok
16:49:06 <irl> that is easier to arrange
16:49:19 <gaba> heh :)
16:49:41 <karsten> let's keep the prague thing in mind, just in case we need it.
16:50:10 <karsten> alright, are we done for today's meeting?
16:50:16 <irl> one final topic
16:50:23 <karsten> yes?
16:50:25 <irl> https://op-ab.onionperf.torproject.net/
16:51:03 <irl> i think we can add this to collector
16:51:12 <karsten> okay. I can do that.
16:51:19 <irl> awesome
16:51:26 <irl> that is all
16:51:28 <karsten> maybe a quick ticket?
16:51:33 <irl> i'll do that now
16:51:35 <karsten> mostly for documentation purposes.
16:51:40 <karsten> great. I'll add it then.
16:51:48 <karsten> great!
16:52:02 <karsten> talk to you next week! bye. :)
16:52:08 <irl> bye!
16:52:10 <karsten> #endmeeting