17:00:24 <hellais> #startmeeting OONI weekly dev gathering 2016-05-09
17:00:24 <MeetBot> Meeting started Mon May  9 17:00:24 2016 UTC.  The chair is hellais. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:00:24 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
17:00:37 <hellais> hello Oonitarians!
17:00:42 <hellais> who is here?
17:00:50 <agrabeli_> hello :)
17:01:25 <willscott> hi
17:02:40 <hellais> to speed up the initial report-back phase we moved that to a pad so we don't clutter your backlog. If you are curious to see what people have been up to you should check it out.
17:02:54 <willscott> link?
17:03:01 <hellais> #link https://storm.torproject.org/grain/xLzNzE64JiHuMERL25Gkrq/ report backs for 2016-05-09
17:03:19 <hellais> feel free to add anything you did ooni related to that pad :)
17:03:28 <catfish99> hi all
17:04:03 <andresazp> Hi
17:04:27 <hellais> catfish99, andresazp: hey there :)
17:04:39 <hellais> so let's begin with the first item on the agenda
17:04:48 <hellais> #topic Come to an agreement on what should be the canonical place to store issues
17:05:14 <anadahz> hello
17:05:30 <hellais> I have a strong opinion for us to switch to using github entirely for the following reasons:
17:06:08 <hellais> * It's tightly integrated with the code review process so it makes it easier to work between issues and code review (pull requests)
17:06:53 <hellais> * A lot of other projects that are part of the OONI ecosystem already live there and are unlikely to move their issues elsewhere (I am thinking of measurement-kit/*, citizenlab/test-lists)
17:08:05 <aagbsn> it's possible to clone/port github issues to gitlab/something else if github ever went goodbye? (I think that's true)
17:08:14 <hellais> * Working with github is much more pleasant than working with trac
17:08:36 <willscott> aagbsn: there's an api to get issue contents, and scripts to do migration.
17:08:55 <willscott> i'd be happy to see github be the canonical place for issues
17:09:36 <agrabeli_> me too
17:10:07 <anadahz> I don't think that we should use Github as the canonical place for anything.
17:10:11 <agrabeli_> other than the obvious free software reasons, what are the advantages of moving everything to trac in comparison to github?
17:10:12 <hellais> aagbsn: I have a script for migrating issues from github to trac: https://github.com/hellais/gh-to-trac
17:10:49 <anadahz> * Github has been banned quite some times repositories and projects.
17:11:24 <anadahz> * It's much easier for an authority to make Github block a project
17:12:36 <aagbsn> it's easier to block *.torproject.org, with little undesirable collateral impact
17:12:45 <willscott> i dunno if i agree with that argument. ooni seems very unlikely to be blocked - there's no copyright infringement going on, which is the only thing github has really complied with afaik
17:12:54 <willscott> and github is available in many more countries than trac is
17:13:17 <anadahz> * It requires that all contributors should create a Github account
17:13:22 <hellais> yes I was about to say that. It seems to me the blocking issue is more relevant for trac.torproject.org
17:13:41 <hellais> just as an example I can't use trac.torproject.org from my university, because they blog *.torproject.org
17:13:42 <anadahz> aagbsn: If *.torproject.org is blocked it affects OONI anyway.
17:14:00 <aagbsn> hrm?
17:14:18 <willscott> what does ooni get from torproject.org?
17:14:24 <hellais> anadahz: how does the blocking of *.torproject.org affect OONI?
17:14:44 <anadahz> willscott: Ref: https://github.com/shadowsocks/shadowsocks
17:14:52 <willscott> that wasn't github
17:15:00 <willscott> that was chinese police showing up at the guys door and telling him to take it down
17:15:47 <anadahz> willscott: and this occured to all other forks as well?
17:16:29 <sbs> hi
17:17:05 <aagbsn> I think it's reasonable to have a few ways to report an issue, such as an email + gpg key, as well as github, but to expect that most development would happen on the default platform of choice
17:17:08 <hellais> anadahz: there are still many live forks of shadowsocks: https://github.com/RockerFlower/shadowsocks.
17:17:13 <willscott> anadahz: like these forks: https://github.com/shadowsocks/shadowsocks/network/members
17:17:43 <willscott> i feel like github handled the shadowsocks thing pretty well. same with the greatfire debacle
17:17:54 <anadahz> willscott: why can you then download it here: https://shadowsocks.org/ ?
17:18:08 <willscott> set up by a 3rd party afaik
17:18:13 <hellais> anadahz: yes I agree that we can provide some other options for reporting issues for people that don't want to use github.
17:18:20 <hellais> *aagbsn
17:18:23 <aagbsn> there was some shady stuff going on with regard to nytimes reporting on that greatfire incident though, and github did 503 the pages
17:18:42 <aagbsn> 'nothing to see here'
17:20:33 <willscott> so as i see there's a trade-off here between control / ownership and accessibility
17:21:05 <hellais> aagbsn: IRC that happenned when china started doing a DDOS to the greatfire pages, so they had to blackhole them for a while.
17:21:22 <willscott> since the downside to 'something bad happening' is that we have to take someone's local version of the repo and put it somewhere else, that transition seems acceptable to me
17:21:29 <aagbsn> yep it looked like a unicorn tarpit
17:21:37 <anadahz> hellais: many scripts use *.torproject to install ooni-*  required dependencies
17:21:54 <willscott> the arguments against delegating it seem to be the moral objections around needing a github account, and the bad-taste of proprietary code
17:22:04 <hellais> tbh I am much more for addressing the imminent issue at hand, having a good issue tracker that is the canonical place to file issues related to OONI, rather than finding a solution for some hypothetical not well defined threat.
17:23:14 <aagbsn> willscott: agreed, that's my only real issue here; has anyone looked at platforms that are better than github in any dimensions?
17:23:41 <sbs> late to the party, anyway my opinion is that I'd rather use github.
17:23:50 <willscott> issues will likely continue to still be reported out of band - email, irc - and existing developers will be able to post those as issues on whatever platform we use, so the overhead of needing an account wherever seems not overly critical
17:24:35 <sbs> regarding the possibility of submitting issues and patches, perhaps ooni-dev could by used by whom does not want to use github? (i.e. one could send emails describing issues and git send-email patches?)
17:24:38 <anadahz> I don't want to support a company that treat employees in a bad way (sexual harrassment, discrimination)
17:24:51 <willscott> i guess my main thing is that i'd prefer not to have (a) another account to remember (b) a new process to learn
17:25:01 <willscott> and i guess high availability
17:25:31 <willscott> so like, us hosting a gitlab instance seems worse to me because i don't want to maintain it and i don't believe we'll have the time to keep it up and running reliably. i'd really rather someone else deals with it
17:25:44 <sbs> willscott: +1
17:25:48 <hellais> willscott: +1
17:26:07 <andresazp> Not that I think my opinions should have any significant weight in the decition, but in my opinion github makes it easier for pleople delopying ooni (like myself) that are less involved than key contributors to participate more
17:26:31 <agrabeli_> willscott: +1, esp since I've never even used trac
17:26:54 <anadahz> sbs: OK, shall I start using ooni-dev for submiting patches and discuss issues?
17:27:12 <anadahz> Will the rest of the OONI team be happy with that?
17:27:14 <andresazp> deploying*
17:27:50 <sbs> anadahz: well, if I see a diff floating on ooni-dev I'm definitely going to read it and provide feedback
17:29:14 <anadahz> I think using email is not the best alternative of that but what I want to say is that people that don't want to open a Github account will not send an email with a patch.
17:30:08 <willscott> i don't think we're going to get any more consensus continuing this discussion now.
17:30:27 <sbs> anadahz: is there actually people that do not have a github account and that would like to contribute to ooni, or is this just a hypothesis?
17:30:33 <willscott> i'd propose the following: for those opposed to github, can you get a working alternative that others are willing to buy into
17:31:02 <willscott> otherwise we'll inevitably have a continued stream of issues coming in through github.
17:31:27 <aagbsn> and no one else will run a gitlab for you?
17:31:45 <aagbsn> (if the issue is, maintenance, vs free software)
17:32:20 <willscott> i bet someone would
17:32:34 <willscott> but there's a bunch of work there
17:32:39 <aagbsn> commercially, even
17:32:41 <willscott> is someone volunteering to do that work?
17:32:56 <willscott> because then we have 3 places where the code / issues could live :-/
17:32:57 <hellais> aagbsn: I think gitlab is not desirable for the reasons we have already mentioned before: * It's one more piece of infrastructure to run, that is likely to not be as reliable as github * It's one more place where you need to create an account * Other projects that we work with are not there
17:33:08 <anadahz> sbs: yes there were some people that's why OONI moved the tickets from Github to Trac two times already hellais can give more details on that
17:34:17 <hellais> anadahz: actually moving it to trac turned out to not be a good move in the end. In the time when we moved to trac from github, it was mainly on ethical grounds and the hypotetical "people don't want to use github" argument.
17:34:57 <anadahz> sbs:  a time where there were a bunch of OONI contributors
17:36:35 <anadahz> sbs:  not only in terms of code but in general , enthusiasts, people that explore things, ...
17:37:10 <hellais> when the issues were moved to trac and issues on github disabled I didn't really see that much increase in reported issues, while it did seem to go up a little bit when I reopenned them in github. Though this is a very un-scientific eyeball measure.
17:37:22 <anadahz> sbs: with lively IRC discussions
17:39:09 <agrabeli_> aren't most developers who would contribute to ooni most likely using github anyway? In terms of non-developers, to my understanding the plan is to provide a GUI through which they can contribute to test-lists
17:39:35 <hellais> anadahz: what period are you referring to exactly?
17:40:29 <hellais> anadahz: also I fail to see how this is relevant to the question of trac vs github.
17:40:32 <willscott> okay. we've spent 40 minutes on this. we should continue with other agenda items rather than rabbit-holing?
17:40:42 <hellais> willscott: yes, indeed.
17:40:45 <agrabeli_> willscott: agreed.
17:40:57 <hellais> can we say that we have reached a consensus that we should be switching to github entirely?
17:41:15 <willscott> hellais: i don't think so. anadahz and aagbsn dissent
17:41:34 <willscott> but we should continue out of band. we don't need to take up all of our weekly meeting on this
17:41:36 <anadahz> sbs hellais:  we can discuss this after the meeting
17:41:43 <hellais> ok
17:41:48 <willscott> we are not likely to get more consensus in the next 20 minutes.
17:41:49 <anadahz> yes let's continue
17:41:53 <aagbsn> i don't dissent - just want to see if we've explored other options than trac/github
17:42:07 <aagbsn> i think github is fine given the reasons mentioned and that we can ditch it if we change our minds
17:42:28 <hellais> aagbsn: ok thanks for clarifying
17:42:33 <hellais> though let's move on
17:42:36 <hellais> #topic Ensure that OONI git repositories are mirrored and synced in different git servers
17:42:37 <sbs> anadahz: ok
17:43:13 <aagbsn> sounds good - should be a hook on each developers with push access? but how does this integrate with pull requests... hmm
17:43:22 <hellais> aagbsn: yeah that is what I was thinking
17:43:28 <willscott> #link https://trac.torproject.org/projects/tor/ticket/18606
17:43:59 <hellais> the only reason why I haven't yet implemented a git hook for that though is that you can't force push to torproject git mirrors
17:44:14 <aagbsn> that is a feature
17:44:17 <hellais> and I generally prefer to squash and rebase commits with master before merging them
17:44:20 <sbs> hellais: what about automatically sync-ing up master?
17:44:23 <willscott> are we regularly force-pushing?
17:44:30 <hellais> so probably it should only happen when pushing to master
17:44:34 <willscott> that seems like it should be an extraordinary event, and be worthy of extra effort
17:44:35 <hellais> willscott: only on feature branches
17:45:12 <hellais> but on feature branches it does happen regularly, because it's generally ideal to not litter the commit history with merge commits
17:45:24 <aagbsn> can you not delete a feature branch?
17:45:28 <aagbsn> (on tpo infrastructure)?
17:46:07 <willscott> do feature branches need the same level of replication as master?
17:46:31 <anadahz> We should also make sure that all OONI related git repositories are synced
17:46:39 <hellais> aagbsn: no you can't
17:46:57 <hellais> willscott: no feature branches don't need to be replicated
17:47:53 <aagbsn> well, sometimes feature branches may be long-lived before merge. it would be a shame if something happened to them
17:49:50 <hellais> aagbsn: yes I guess, so but I really don't want to loose the ability to be able to clean up history before merging into master.
17:50:14 <willscott> they're already going to be in at least 2 places right? on the developer's machine and wherever the developer has chosen to make them public
17:50:34 <sbs> willscott: yep
17:50:37 <willscott> syncing of those seems needed only if we have multiple collaborators on a feature branch pushing to different replicas
17:50:44 <hellais> another thing to consider is that a lot of the merges are done via the github interface, this means that a local git hook would not be sufficient.
17:52:03 <willscott> hmm. so what more do we want?
17:52:49 <anadahz> willscott: currently the git repos hosted in torpoject are not in sync
17:53:34 <anadahz> either we decide to rip them and host a git backup somewhere else, or we keep them in sync
17:54:44 <anadahz> but having git repos that are 19 months unsynced it's not desireable
17:55:13 <willscott> seems reasonable
17:55:44 <willscott> so, the issue as i see it is that access on the tpo repos is limited based on some credentials that are not clear to me
17:56:54 <willscott> but we can have a webhook from github that triggers an automatic pull to tpo with the right credentials
17:56:59 <willscott> is there a vm where that could live?
17:57:35 <willscott> or we could even give github a deploykey to auto-push directly to tpo, i think
17:57:57 <willscott> maybe that's the other direction though
17:57:59 <anadahz> willscott: perhaps this gitlab instance needs to be created ;)
17:58:12 <willscott> then we have 3 things to keep in sync :p
18:00:29 <willscott> anadahz: i have no strong objection to such a thing existing, but hypothesizing about it isn't going to be productive. if this is the direction you think is right, propose a place for it to live, and who's going to do the work of setup and maintenance.
18:03:28 <anadahz> willscott: I'm just proposing it as an option no need for 3 way syncing, backup git repos can exist there together with the gitlab instance. I don't think that maintaining the Gitlab instance will be that much work since we 'll need to maintain the backup git repos anyway
18:04:08 <willscott> who's volunteering to do it?
18:04:09 <anadahz> except if we find a viable solution with the torproject git repositories
18:05:24 <anadahz> I can volunteer to do that together with the person that will setup the backup/sync git repos process
18:07:10 <willscott> who has access to the tpo repo besides you and hellais?
18:07:53 <anadahz> willscott: I dont have access to the tpo repos.
18:07:58 <hellais> willscott: isis and aagbsn should also have access to them.
18:08:19 <hellais> anadahz: you don't?
18:08:30 <anadahz> hellais: no
18:09:16 <willscott> that seems like a pretty good indication that we shouldn't be using those as anything other than mirrored backup
18:10:26 <anadahz> related #17524
18:11:08 <hellais> yeah, I think what we should decide is if it's acceptable for them to not be perfectly in sync and them only being synched when I remember to push to them (i.e. not that often as it turns out, though I can try to make an effort, but knowing myself I will sometimes forget to synch them and could be off by some weeks) or if we need this sync process to be automatic
18:12:40 <willscott> hellais: if the goal is to have a backup, then months out of date probably doesn't do that
18:13:13 <willscott> i guess i see 2 ways forward:
18:13:41 <willscott> anadahz sets up a gitlab. that becomes backup initially or maybe eventually primary. we get rid of the tpo ones
18:14:43 <aagbsn> i dont think we should have official repos that aren't sync'd
18:14:55 <willscott> or hellais gets an auto-pull to happen to tpo or figures out a way to get credentials for someone else to do that
18:15:46 <aagbsn> the process should be automatic, or a pull/push script a developer with access can run. github should not get to push to tpo automagically, though at least it cannot force push
18:16:07 <hellais> willscott: I will look into a way to make it so the sync will happen automatically from my dev machine.
18:16:49 <willscott> aagbsn: i was imagining a sync run on a developer machine that was triggered by a github webhook
18:17:23 <aagbsn> isn't this just git pull github master && git push tpo master ?
18:17:34 <willscott> sure is :)
18:18:05 <willscott> hopefully takes about the same time as the 40 minutes we spent talking about it :)
18:18:09 <aagbsn> what's hard about doing that periodically :)?
18:18:10 <hellais> aagbsn: yeah the command is trivial, it's just that if some things are merged via github and I forget to run it for a while, these issues with it being out of sync happen
18:18:27 <willscott> a cron job also seems totally fine
18:18:34 <aagbsn> it only breaks if someone force pushes to master :)
18:18:41 <anadahz> hellais:  anacron ?
18:18:58 <hellais> yeah I will figure this out. I don't think we should continue this discussion
18:19:14 <willscott> great. should we tackle another agenda item this meeting?
18:19:22 <willscott> #action hellais will figure out syncing to tpo
18:19:33 <anadahz> I think we are a bit overtime
18:20:11 <hellais> yeah I think at this point we should defer them to next week
18:20:15 <anadahz> and I would like to listen from andresazp and catfish99 if they have something to mention/comment/report back
18:21:01 <andresazp> On the VE deployment?
18:21:24 <willscott> sure. anything since last week or things you'd like from ooni?
18:22:12 <catfish99> since last week - just that updated VE testing list was shared with the OONI team. request that data be merged with existing testing
18:22:35 <catfish99> so that future testing will include new/updated sites that were identified
18:22:45 <andresazp> I havent checked the conversation last week, I´m not working closely with catfish99
18:22:59 <andresazp> I added links to reports on the document linked i the email
18:23:03 <hellais> catfish99: so can we publish the testling list? I was under the impression you wanted us to wait for the report to go public
18:23:19 <andresazp> Hi catfish! this is andres
18:23:20 <catfish99> wait until report goes public - later this month
18:23:34 <willscott> sounds good.
18:24:34 <agrabeli_> sorry guys I really need to leave, but happy to discuss any pending issues later on jabber
18:25:21 <hellais> catfish99: ack
18:25:22 <andresazp> I undestand that´s happening next week
18:25:43 <willscott> great. lets end?
18:25:53 <andresazp> Question
18:26:00 <willscott> go for it
18:26:05 <anadahz> catfish99: we still don't have the translated version
18:26:21 <andresazp> the topics like depricating DNS_consistency, etc are pushed for the next version?
18:26:29 <willscott> yeah. until next week
18:27:06 <andresazp> andadahz: you meen the url test lists?
18:27:52 <hellais> anadahz: yes we should talk about that next week, but very briefly I can tell you that we have been developing a new test that will be testing for most of the things that http_requests and dns_consistency does, except it's a bit more accurate.
18:27:53 <anadahz> andresazp: no it was about the report text
18:28:11 <hellais> this is the test: https://github.com/TheTorProject/ooni-spec/blob/feature/web_connectivity/test-specs/ts-016-web-connectivity.md
18:28:20 <hellais> andresazp: ^^
18:29:49 <hellais> if we don't have anything else I guess we can end this?
18:30:19 <andresazp> I can send detailed info on how the testing was peformed
18:30:41 <andresazp> and the incidents identified, the text of the report itself is out of my scope for this onw
18:30:43 <andresazp> one
18:31:52 <andresazp> I´m very keen in participating on the discussion about the web connectivity test next week
18:32:53 <hellais> andresazp: excellent :)
18:35:07 <andresazp> Thankfully recent timzone changes in VE make it much easier for me to login to the weekly meetuo
18:36:00 <anadahz> andresazp: You can always sen an email to ooni-dev mailing list (https://lists.torproject.org/cgi-bin/mailman/listinfo/ooni-dev) and continue/start the discussion from there.
18:40:22 <anadahz> I think we are good to end this meeting?
18:40:52 <hellais> sounds good
18:40:54 <hellais> thanks for attending
18:40:57 <hellais> #endmeeting