17:01:25 #startmeeting trac->gitlab October 15th 17:01:25 Meeting started Tue Oct 15 17:01:25 2019 UTC. The chair is gaba. Information about MeetBot at http://wiki.debian.org/MeetBot. 17:01:25 Useful Commands: #action #agreed #help #info #idea #link #topic. 17:01:33 We have an agenda here: https://pad.riseup.net/p/e-q1GP43W4gsY_tYUNxf 17:01:45 please review 17:02:07 looks good 17:02:27 same 17:03:32 ok, it seems that it may be just the 3 of us today :) 17:03:40 * catalyst is here 17:03:52 hi catalyst! o/ 17:03:59 hi! 17:04:18 ok, 1st topic is review the registration of users (mail I sent to the tor@) 17:05:05 the proposal is to have gitlab as it is and registration open (with a team moderating it) 17:05:16 so far geko, pili and me offered to moderate gitlab users 17:05:24 count me in there too 17:06:17 if we see that is too much for us to handle then we close registration and get people to send a mail or similar for us to create accounts 17:06:30 anybody disagreeing with this as a start? 17:06:47 not me 17:06:53 * ahf is good with it 17:07:06 catalyst? 17:07:15 is this with or without namespace suffixes? 17:07:48 not sure if we can have users created with -guest automatically when they register 17:07:49 (not a significant factor for me personally, but other people have said stuff about it before) 17:08:01 we have been creating external users with the -guest 17:08:02 * GeKo is somewaht here but needs to work on the otf proposal 17:08:55 we will see if they can be created with a suffix automatically 17:09:06 otherwise they just get to be users with any name 17:09:10 i think we need something like the salsa sign up app for that 17:09:40 * anarcat concurs 17:10:40 ok 17:10:41 * ahf is checking 17:12:04 nope, looks like we cannot define a suffix in GL itself :-) 17:12:16 ok 17:12:17 but, we can enable some lists of domain names we like for email. that is worth having in mind in case user spamming gets bad 17:12:32 yes, whitelisting and blacklisting domains I think is a good idea 17:12:36 ye 17:13:30 it seems we have all issues resolved for this migration 17:14:29 the document for the migration is up to date to what has been discussed and resolved at previous meetings 17:14:30 yes \o/ 17:14:36 anybody has any question about it? 17:14:49 * anarcat goes to check the doc again 17:14:50 or about what was resolved before or any doubt about anything? 17:15:34 are we going to continue looking into using Discourse? 17:15:50 as far as discourse is concerned, this is kind of on hold 17:15:56 not in this specific proposal of migration. I think we should but it is a longer term project 17:16:09 yes, on hold inside this migration project 17:16:11 we were hoping the blog discussion would spur this forward, but the short-term stuff on that front is to fix things up in the blog first 17:16:40 so maybe it will move ahead in the future, but it shouldn't be a blocker for the migration, otherwise it would mean waiting a possibly long time for the migration 17:16:57 in other words, discourse is punted over the web/blog people, afaik 17:17:03 ok 17:18:30 i'll mark the irc number bot as "solved" insofar as it needs work but we have a plan 17:18:44 ok 17:20:47 what's our solution with trusting gitlab again? 17:20:58 we talked about keeping gitolite, but i'm not sure that's viable in the long term 17:21:19 i have an idea about irc bot and integration in general but i want the ticket migration stuff to be done first 17:21:38 okay 17:21:38 gitolite is the back end for the read-write git repositories? 17:21:39 in the short term we will continue using gitolite 17:21:48 catalyst: no, gitlab has its own git shell thing 17:22:00 and I suppose some projects will move to gitlab for code too 17:22:01 anarcat: i meant what we're using gitolite for right now 17:22:07 catalyst: the concern is we might not want to trust gitlab with our most important code because the attack surface is larger 17:22:17 catalyst: yes, we use gitolite right now 17:22:37 i'm talking about "Feature Comparison between Trac and Gitlab 17:22:47 and I assume code will be mirrored into gitlab for all projects 17:22:48 in "Code Management", "Git" 17:22:57 but we still use gitlab for code management 17:23:04 "Some people have concerns with using gitlab to merge code into some repositories that could be more sensitive." 17:23:13 "Possible solution: to maintain some repositories in gitolite." 17:23:26 isn't this tor.git only? i don't think i have heard this concern mentioned for other repos? 17:23:31 but okay, if "keep gitolite for some stuff" is the solution, i guess that's okay 17:23:34 yes, i think the migration of code should be up for the mantainers 17:23:37 it's a concern for tpa 17:23:39 anarcat: right. i've heard that before. i'm wondering what the threat model is here 17:23:40 oh! 17:23:50 makes sense for tpa 17:23:51 i'm not sure how to frame this problem 17:23:53 tpa and nick had this concern 17:24:04 i think the solution for this is to sign commits and auth with pgp 17:24:05 as far as i know nothing sensitive deploys directly out of git.tor, right? 17:24:06 on both ends 17:24:20 i don't know 17:24:38 i must admit i'm not super familiar with the ways of git.tpo 17:25:21 the document still says we're going with a cypherpunks account, so that might need changing? 17:25:38 oh, didn't we have some sticky stuff with labels too? 17:26:06 oh yes 17:26:16 let's change that in the doc 17:26:21 sticky stuff? 17:26:32 i don't remember, ahf didn't you say you had some trouble with labels? 17:26:37 maybe in the categories migration? 17:26:39 i don't remember 17:27:00 no, that was just a nice to have feature in the enterprise version 17:27:11 ah 17:27:22 well good, no problem, i like those :p 17:27:27 i finished reviewing the docs 17:27:28 * anarcat concurs 17:28:35 ok, anybody else need more time to review doc? 17:28:51 * ahf is good 17:29:27 let's talk last about dates 17:29:35 at the end of the doc there are steps for this migration 17:29:38 yes :-) 17:29:50 where it says 'MIGRATION’S PLAN 17:29:52 ' 17:29:59 Are we ok with those? 17:30:30 what is 4.b)? 17:30:33 Migrate team’s wiki documentation repository for the group. 17:30:57 We need to migrate trac's wiki too 17:31:02 from what i understand, we only have one wiki now, does that mean we split the trac wiki in the migration? 17:31:09 * catalyst realizes people are looking at a different nextcloud doc 17:31:12 4b is very open, i have no idea how we are migrating that 17:31:17 right okay 17:31:20 right 17:31:23 so i'd move that up one level 17:31:39 say point 3e) or 3bis: migrate wiki in legacy project 17:31:42 i have the feeling wikis are something where we care less about the history than everything else though? 17:32:04 my rationale is that we have one wiki, and we migrate it into trac, and it's already a huge deal because we need to handle markup (and maybe history) and all that stuff 17:32:14 so your proposal is to migrate the whole wiki into the project 'legacy' 17:32:15 then, like tickets, we can split it up later if we need 17:32:18 yeah 17:32:25 ok 17:32:32 i think we underestimate the complexity of this bit, to be honest :) 17:32:37 the wiki is vast and old and venerable 17:32:50 there's a lot of shit in there, and it doesn't map super well to what gitlab offers 17:32:59 because right now we basically treat it as a website 17:33:01 how much cross referencing are we willing to lose from the Trac wiki, if any? 17:33:05 but gitlab offers *many* wikis 17:33:10 like one per project 17:33:13 so we need to think about how that will work 17:33:23 my argument is we can punt this to "later" if we move it to legacy 17:33:48 i think we can ask teams to migrating their stuff over? and we can ask vegas leads what they think we should move over for the global side of the org? (like our meeting list page) 17:33:49 there is a tool (that anarcat suggested) to do migration from trac into gitlab. I was hoping we could use that for the wiki 17:33:50 because then #links will still work and we will do the hard work of parsing the textile or creole or whatever langguage trac uses (i forgot, to be honest) into markdown 17:34:14 i think most of the network team wiki pages could use having some manual eyes looking at them 17:34:15 ahf: the wiki isn't split up into teams, it's not realistic to expect a team-based approach will migrate everything 17:34:16 I'm up for moving the wiki into legacy and then spliting after 17:34:21 too much stuff will be left on the wayside 17:34:30 anarcat: no, but some of the pages that have quite a bit of activity is team pages 17:34:39 gaba: i'm hoping tracboat or bits of it can help us do the conversion 17:34:42 ahf: +1 17:34:52 ahf: sure, but that doesn't mean the rest isn't useful :) 17:35:09 and there are some "global" pages like, as you said, meetings or https://trac.torproject.org/projects/tor/wiki/org/operations/services 17:35:10 we will need a place for team wikis and a place for the rest of the org 17:35:13 no, but isn't that something we can solve on an on-demand basis and migrate things over as needed? 17:35:20 i mean just look at this https://trac.torproject.org/projects/tor/wiki/org 17:35:29 anarcat: isn't that a page maintained by the tpa team? 17:35:30 ahf: no, i don't think it is :) 17:35:35 ahf: hahaha 17:35:38 ahf: haha. no. :) 17:35:52 it's maintained by Nobody™, AFAIK 17:35:57 so if nobody maintains the services page, then it should just go away 17:36:04 or some team that cares for it can start maintaining it 17:36:19 i'm sorry, but i don't think we can just take a scorched earth approach here 17:36:36 it's not because the current wiki is badly maintained or orphaned that we should just leave it to rot there 17:36:37 migrating things that nobody owns is not a blocker IMO. 17:36:52 well that's why i'm proposing we migrate the wiki as a whole 17:36:58 i don't understand what the alternative is, actually 17:37:01 so it can rot there :-) 17:37:04 i'd be happy to hear another proposal 17:37:08 well 17:37:15 ahf: so it can rot in the migrated "legacy" wiki space? 17:37:16 "not migrating the wiki at all" is a proposal, sure 17:37:18 that is why i don't think it is a blocker. i'm fine with us doing it, but i don't think it should block anything :-) 17:37:26 i think it's a blocker 17:37:27 ah, no, sorry, i am not saying we shouldn't move things over 17:37:38 okay, maybe we need to sync up :) 17:37:45 i propose we migrate the wiki over to gitlab 17:37:50 into the legacy project's wiki 17:38:07 that process converts the markup as best as it can, keeping legacy links and things working as much as possible 17:38:10 +1 to move the whole trac wiki into gitlab legacy project's wiki 17:38:16 and marking stuff as Ticket macros as broken or something 17:38:27 the ticket list macros? 17:38:29 the wiki will need some work 17:38:30 then *later*, some nice sunny day, people pick up bits and pieces and move them to the right places 17:38:41 ahf: stuff like the lists in here https://trac.torproject.org/projects/tor/wiki/user/anarcat 17:38:48 that will die in the migration 17:38:49 anarcat: ok i like this plan as it seems a "least bad" option :) 17:38:51 we need to handle that somehow 17:38:52 yeah 17:38:57 yeah, those tables 17:38:58 i mean i'm happy to hear another plan :) 17:38:59 cool! 17:39:11 but i don't like the "let it rot in trac" plan 17:39:15 teor also has a page that uses quite frequently https://trac.torproject.org/projects/tor/wiki/user/teor 17:39:16 but maybe i just misunderstood :) 17:39:17 anarcat: are you up for looking into how we can use the tool that needs to dive into the db itself and do the migration of the wiki with that? 17:39:20 then we can do that in parallel? 17:39:35 ahf: well i looked into tracboat for that, and it has a converter to markdown 17:39:36 i have forgot the name from top of my head right now and it's on my laptop and i'm on my desktop right now 17:39:42 yes! the tracboat tool! 17:39:43 tracboat? 17:39:44 yeah 17:39:45 yes 17:39:45 anarcat: it would be awesome if you can be in charge of the migration of the wiki 17:40:03 well it's a bit tricky, because tracboat is this kind of all or nothing thing 17:40:12 you give it a trac install and it spurts out a gitlab instance 17:40:20 so it does tickets and wiki and everything 17:40:24 not sure we can just pick the wiki out of that 17:40:34 short of rewriting yet another conversion tool 17:40:37 can we hack it a little to only do wiki 17:40:42 maybe? 17:41:05 may i remind people here that was one my objections to writing our own converter in the first place? :p 17:41:18 i guess that's not very useful ;) 17:41:25 i can remind you that tracboat didn't do everything we wanted for ticket migration 17:41:28 in case that was forgotten 17:41:33 :) 17:41:37 ugh, i guess i'm relunctant to digging into another development project, i always feel like i have my hands full 17:41:53 ahf: maybe we shouldn't get back in that debate then :) 17:42:04 * anarcat retracts his poorly placed reminder 17:42:11 anyways 17:42:18 i can look into how much work i think it would be - i think wiki's are less work than tickets 17:42:32 but i have no idea about all the references inside the wiki 17:42:35 that is the only concern i have 17:42:36 ahf: i seem to recall i sent you links to the tracboat code for the parser 17:42:41 yeah, it's kind of a problem 17:42:42 i have already stolen the markdown converter from tracboat 17:42:46 oh you did 17:42:49 yes 17:42:58 well i guess that's the magic bit you need 17:43:01 it is good, required some modification, but it was easier than rewriting it myself 17:43:06 i knew i looked at this before :) 17:43:22 yeah, for tickets and for wiki. but i think i am not sure how to rewrite internal references, but i can look into that on friday 17:43:25 shouldn't take too long 17:43:37 okay 17:43:42 well, if you get stuck do ping me 17:43:47 i will for sure :-D 17:43:55 i can do a status to you on friday i think 17:43:59 i would rather stand a little aside on the tech work here, as far as programming the converter is concerned 17:44:09 i feel you have started this project and i can help 17:44:12 my last two fridays have been trying to run the converter on all tickets up to 30k and find bugs in it 17:44:19 but i feel like starting a second converter effort would be messy 17:44:22 and that takes a bit of time where i can look into other stuff 17:44:31 cool 17:44:32 anarcat: sounds fair to me! 17:44:33 cool 17:44:39 https://www.xkcd.com/303/ 17:44:52 ahahaha, i have that as a t-shirt 17:45:05 gaba: should we try to think of some dates 17:45:08 but yeah, i can definitely work on the systems sides, although hiro has a better grasp of that than me 17:45:20 where is hiro, actually... shouldn't she be in those meetings? :) 17:45:32 yes 17:45:33 my "internal dates" right now have been: have a "full" (for full being everything that i haven't forgotten) migration to legacy/ on gitlab by medio november 17:45:36 next is dates 17:45:42 and then have two weeks of asking everyone to find mistakes in that by 1st of december 17:45:59 and then if we don't need to bull any brakes here, do the move somewhere early december 17:46:16 I am here 17:46:27 hi hiro o/ 17:46:31 ahf: that sounds good 17:46:34 hey hiro :) 17:46:38 I need to read backlog 17:46:39 mid november to the test 17:46:57 hiro: i have been trying to do a summary of discussion in the pad https://pad.riseup.net/p/e-q1GP43W4gsY_tYUNxf 17:47:04 yes, i think there is buffer there for a catastrophe for ticket migration and i haven't found any yet 17:47:07 so there is space for the wiki thing too 17:47:08 ahf: early december sounds a bit tricky to me because it might mean "oh well, it's mid-december already" and next thing you know, you're running the final migration on christmas eve 17:47:15 and you spill gravy all over gitlab 17:47:28 and then you're sad because you spoiled both gravy and the website ;) 17:47:34 and the question ahf, is how we can help 17:47:35 anarcat: i have no worries with postponing it if that is needed - i don't want to ruin people's christmas if they celebrate that 17:47:55 gaba: when the move has happened, i really need everyone (not just peopel in here, but everyone in the org) to help look for errors in the migration 17:47:56 ahf: well, celebrate newton's birthday or whatever, the point is people go on vacations during that time ;) 17:48:01 oh wow we want to open gitlab to the abysses of spam? :D 17:48:02 anarcat: yes! 17:48:05 and if you want a vacation, you don't want to start a migration a week before that :) 17:48:12 i also go to the ccc thingy after christmas 17:48:15 hiro: i figured you might want in ;) 17:48:24 ahf: madness ;) 17:48:30 hiro: how long it would take to have salsa with gitlab? 17:48:31 yeah I am going to CCC too 17:48:52 I haven't looked at the registration app that debian use 17:49:07 I suppose it's somewhere in the playbooks too 17:49:36 i think i saw it in the playbooks and disabled it there (was my first experience with ansible so i might have messed something up) 17:50:03 yes there are a few things there in the playbooks 17:50:12 (also gitlab pages and so... ) 17:50:23 yeep 17:51:38 do you have link to the playbook? 17:51:43 s 17:52:15 https://gitweb.torproject.org/admin/services/gitlab/dip.git/ 17:53:01 ok, anything else that we need to consider? 17:53:24 i don't think we had a conclusion to dates? 17:53:32 if adding salsa is a very quick thing then i think we should do it... 17:53:40 i would think most people leave for holidays around the 22nd of december or something like that 17:54:04 ahf: I though you said mid November. We could do the tests before holidays and do the final migration when we come back. 17:54:09 so aiming for being able to migrate the first week of december and if anything is delayed quickly postpone it to early january sounds reasonable? 17:54:20 yes, that sounds good 17:54:25 mid november for having the migration done to legacy/, then ask everybody to find errors 17:54:29 and fix those errors 17:54:36 then do the migration early december 17:54:43 and hopefully no errors will be there, but that is a lucky shot 17:54:46 and we are humans 17:55:08 sounds ok to me 17:55:22 i concur 17:55:29 being human sounds okay 17:55:32 catalyst, hiro: sounds good? 17:55:32 ;) 17:55:44 who is going to migrate the tickets in the end? 17:55:51 will that be team responsability? 17:55:52 me i think? 17:55:55 hiro: ahf: is working on a script 17:55:58 ah ok 17:56:03 let me know if I can help 17:56:08 and everything gets moved to a legacy/ project 17:56:13 so we can see what hasn't been migrated 17:56:19 and move that afterwards if we decide to create projects for it 17:56:22 like more specific projects 17:56:43 hiro: as far as tpa is concerned, i think there will be some work to change our workflow when/if we switch away from git.tpo to gitlab 17:56:44 gaba: sounds ok to me as a rough timeline 17:56:55 i mean all teams will need to adjust 17:57:00 but we are special in so many ways 17:57:02 uhm are we also switching from git.tpo? 17:57:05 * ahf keeps reading 'tpa' as 'tor browser android' 17:57:21 hiro: i was assuming we would 17:57:28 is everybody comfortable with that? 17:57:31 hiro: are we keeping gitolite forever is another way to ask this question 17:57:32 hiro: not for all repos 17:58:22 anything else? can we close the meeting not then? 17:58:23 actually, the migration plan says nothing about git repos 17:58:26 now* 17:58:32 but maybe we can talk about that another time :) 17:58:46 it seems unclear to me what repos will migrate from git.tpo to gitlab and how and when 17:58:49 when people think we should meet again? 17:59:04 about the repos: I was thinking that is up to the mantainer of the repo 17:59:10 is a different issue than migrating tickets 17:59:18 okay, so it's like jenkins i guess 17:59:21 we get rid of trac 17:59:33 we don't deal with gitolite or jenkins (yet) 17:59:43 and people are free to jump onboard if they want 18:00:04 we should still think about how that will happen though, because urls would break during such a transition and we'll have to provide people support to fix that 18:00:11 but that can be done in another meeting 18:00:14 let's meet two weeks for now and do status on migration 18:00:18 with focus on wiki and tickets 18:00:28 by then we should be able to see something and discuss it and see if our timeplan is OK 18:00:37 i'd like to talk a bit about repos too if poss 18:00:55 makes sense 18:01:03 but 2w seems good to me 18:01:05 October 29th is 2 w 18:01:18 doable 18:01:28 argh, i am traveling that day 18:01:36 29, 30, and 31st i'm in zurich 18:02:02 november 5th? 18:02:05 it could be 3 weeks 18:02:14 or 28th? 18:02:21 * gaba is traveling on the 28th 18:02:26 ahhh :-d 18:02:26 back from mozfest 18:02:29 ok 5 is OK for me 18:02:35 5/11 18:02:38 or 25th? 18:02:43 we could iterate a bit faster 18:02:56 although i guess fridays are when ahf does the most work so it's not the best time ;) 18:03:00 ahf: how do you feel about that? 18:03:08 i would prefer 5/11 over 25 18:03:11 ok 18:03:14 i like my fridays as being less irc :-S 18:03:31 sounds good 18:03:36 5/11 cool 18:03:39 sounds good 18:03:42 #endmeeting