17:58:53 #startmeeting 17:58:53 Meeting started Thu Jul 2 17:58:53 2020 UTC. The chair is paddatrapper. Information about MeetBot at http://wiki.debian.org/MeetBot. 17:58:53 Useful Commands: #action #agreed #help #info #idea #link #topic. 17:59:00 #topic roll call 17:59:04 Agenda: http://deb.li/oNCD 17:59:13 0/ 17:59:14 Please say hi if you're here for the meeting 17:59:19 o/ 17:59:21 hi if you're here for the meeting 17:59:23 I forgot again 17:59:24 hi 17:59:35 (actually not, but hey) 17:59:42 hi 18:00:26 #topic Action items from last meeting 18:00:43 tumbleweed: have you had a chance to create the 4 issues on salsa? 18:00:48 heh 18:00:55 no! I just thought about that yesterday :P 18:01:07 heh no worries 18:01:14 hi 18:01:16 yeah, the issue page is still empty 18:01:20 #action tumbleweed to create issues for the 4 pending action items 18:01:54 #topic Video Stack 18:02:05 pollo: you've been playing around with Vocto? 18:02:10 (no progress on my side at all) 18:02:31 yes! 18:02:36 o/ 18:02:55 so I did sucessfully run the FOSDEM voctoweb frontend 18:03:12 #info pollo successfully ran the FOSDEM voctoweb frontend 18:03:15 I think it works well enough and can be easily customized to for our needs 18:03:29 awesome 18:03:31 my next step would be to re-work their ansible role 18:03:38 yeah, it's a bit ugly 18:03:45 I looked at doing that a while ago, but meh 18:03:46 I spun up a voctocore to play with 18:03:56 so, we should stick a voctoweb to it 18:04:01 I also managed to playback videos to voctomix from the CLI 18:04:04 using "$ voctomix-ingest --video-source file --video-attribs "location=foobar.mkv"'" 18:04:17 worked well, but I didn't check audio 18:04:23 tumbleweed: yes, that's my next step 18:04:32 maybe we should make it possible to control that from the web frontend, too? 18:04:32 the main issue with audio is that it needs to change with the source 18:04:34 having that will let me check for audio synx 18:04:37 which is different from our normal setup 18:04:50 #info we can play video files using voctomix-ingest, but not tested audio yet 18:04:52 ivodd: ah yes, good point 18:05:03 ivodd: that can be a command passed to vocto 18:05:06 #action pollo to test audio sync 18:05:17 it's not very hard to do, we'll need to make presets anyway 18:05:30 #action pollo to add voctoweb frontend to video team's voctocore VM 18:05:34 pollo: if the pre-recorded talks are available, maybe we could have a link in the voctoweb thing to say "start playing the video now"? 18:05:41 wouter: yes, that's the idea 18:05:45 i.e., have a list of videos etc 18:05:46 right 18:05:57 and then that link can try to ensure that audio works, too 18:05:59 olasd volunteered to add a list of available recordings to voctoweb 18:06:08 cool, so that's great then 18:06:15 #action olasd to add a list of available recordings to voctoweb 18:06:16 yeah, I'll look at that during sprint (I took friday off of work) 18:06:25 that's it for me 18:06:31 paddatrapper: #info we got VMs from infomaniak to host our infra 18:06:41 #info we got VMs from infomaniak to host our infra 18:07:09 (and Monday 13/Tuesday 14th are vacation days too, which I may use for sprint overflow tasks :P) 18:07:19 are there any points in that that need mentioning? 18:07:21 tumbleweed: ? 18:07:45 busy fighting with the nvidia card in the one VM with a GPU 18:07:51 haven't got it to work yet 18:08:02 though if we have working Vocto do we need GPUs? 18:08:02 tumbleweed: do we need that if we're going to use vocto? 18:08:05 heh 18:08:06 But, on the flip side, OBS seems to work on llvmpipe, if you throw enough CPU at it 18:08:07 I thought it was only necessary for OBS 18:08:18 wouter: Still keeping options open there 18:08:24 fair enough 18:08:24 I think tumbleweed's tests with OBS are worth it 18:08:42 though I understand it needs a lot more CPU than vocto does? 18:08:43 OBS needs/wants GPU, vocto doesn't 18:08:49 #info GPU instances with nvidia cards don't work as expected at the moment 18:08:52 at least for now. I'd still prefer it if we can use vocto, but options are a good thing to have 18:08:53 if so, I'd be reluctant to use it if we can avoid it 18:08:57 wouter: OBS composites with GL, as I understand it 18:08:59 true 18:09:17 don't get me wrong, I agree that it's good to test 18:09:27 but if we don't resolve the nvidia issues, I'd rather go with vocto than with OBS 18:09:28 #action tumbleweed to investigate OBS on the GPU instance, to keep options open 18:09:31 so, on the infomaniak VMs 18:09:43 even if we can get stuff to work, I would rather have something that doesn't nearly max out our VMs 18:09:53 aside from the GPU one, they seem to be on a private v4 network, with NATed public IPs 18:09:58 and public IPv6 18:10:10 connectivity between them seems reasonable 18:10:22 no reason to think we can't use them for all of our infra, at this point 18:10:23 didn't zigo say it was 10Gbit? 18:10:33 yeah. Although that 10gbit will be shared by other users 18:10:38 sure, but still 18:10:50 are you saying that's good, or bad? 18:10:51 #info infomaniak VMs are on a private v4 network, with NATed public IPs and public IPv6. Good conectivity between them 18:10:58 I'm saying it's definitely not bad 18:11:00 as long as we don't let wouter use all these 10gpbs for sreview ;p 18:11:05 olasd: heh :) 18:11:14 tumbleweed: but we'll keep using the DO droptlets for the streaming frontends right? 18:11:16 olasd: well, I was thinking here, you know... 18:11:21 pollo: may as well 18:11:24 I certainly think we should default to infomaniak where possible 18:11:33 streaming frontends would not work there for example 18:11:53 would we want to spin up a debconf-specific jitsi, there? 18:11:56 it's certainly also a good idea to spread streaming frontends across the globe 18:12:06 #agreed use infomaniak VMs for video team infrastructure where possible 18:12:23 tumbleweed: it would prevent having to mess with the streaming script, so maybe 18:12:27 it would probably be nice to have jitsi close to voctomix if the machine can handle it 18:12:28 the NAT may be an issue though 18:12:39 the NAT doesn't seem to have been an issue, yet 18:12:40 (jitsi and jibri, that is) 18:12:44 I was getting several gbps through it 18:12:57 tumbleweed: speed isn't the issue, webRTC and Jitsi are 18:13:02 I'd guess with some /etc/hosts munging, we can make that all behave 18:13:15 maybe, gave up on it last time 18:13:28 OK, so that's probably a next action here 18:13:32 I'm sure people have managed to run jitsi on public clouds with nated public IPs before 18:13:34 ah yes at AIMS we set up that one jitsi instance on a machine that also has a private IP but a public IP entirely natted to it and jitsi just couldn't do webrtc at all 18:13:36 may just need a proper TURN server 18:13:50 want to action me to setup a jitsi? 18:13:57 I'm happy to 18:14:03 set up enough of them at this point 18:14:14 tumbleweed: if you do have time for that, please have a look and Ansible Galaxy and check if we can reuse a role there 18:14:21 no point in writing tons of generic code 18:14:25 #action paddatrapper to setup a Jitsi VM and add to ansible 18:14:31 yeah, I was about to say, might be a good idea to have an ansible something 18:14:37 they provide a debian package 18:14:42 even so 18:14:45 so any existing ansible role may not have much value on top of that 18:14:46 you still need configuration 18:14:50 it's more complicated then apt install :s 18:14:53 oh, that part :) 18:14:54 right 18:15:19 tumbleweed: do they provide a repository, or just a package? 18:15:24 wouter: repo 18:15:33 you need to manage TLS certs, prosody, multiple services, etc. 18:15:36 * wouter makes a note to add it to extrepo, then :) 18:15:41 but yes, we should look at the ansible stuff 18:15:41 and the debian packages are kinda crap 18:15:51 they break often 18:16:48 Anything else on the video stack, or shall we move on to streaming setup? 18:16:49 anyway, I've been burnt by Jitsi in the past and I'm happy others are in charge of it :p 18:17:01 pollo: :D 18:17:14 paddatrapper: I think we should move on? 18:17:15 it's good to get more people familiar with jitsi 18:17:23 I certainly learned a bit, adding the voip stuff 18:17:30 #topic Streaming Setup 18:17:47 #agreed streaming frontends still on digitalOcean 18:18:00 #action olasd to try minimize stream delay 18:18:21 don't think there is anything else? 18:18:25 we've started butting heads on limitations of the nginx geoip module during MDCO so we need to figure a better solution for that 18:18:29 don't think so either 18:18:33 I think tumbleweed wrote $something 18:18:41 olasd: can you expand on that? what's the issue? 18:18:53 I did 18:18:55 I need to finish it 18:19:08 stilll stuck on the maxmind license 18:19:16 wouter: the geoip module shipped with nginx only supports the old database format, which maxmind has stopped providing free of charge 18:19:25 oh, okay 18:19:28 and the new one has non-distributable terms 18:19:33 so we can't trivially ansible distributing it 18:19:39 #info the geoip module shipped with nginx only supports the old database format, which maxmind has stopped providing free of charge 18:19:44 (see the debian-legal thread) 18:19:47 so, essentially, we should move to something not geoip I guess? 18:20:02 tumbleweed: well it's not that different from the blackmagic situation 18:20:06 yeah 18:20:09 (would be cool if we could have a database that knows about routing distances rather than country...) 18:20:31 well, we don't need to go over all the details in this meeting 18:20:43 tumbleweed: do you want an #action for this? 18:20:45 well, that's called "anyrouting", I guess, but that would get us too far 18:20:47 er 18:20:50 anycast, I mean 18:21:00 paddatrapper: sure, give me an action to finish this 18:21:11 I can also stare at it during sprint 18:21:14 I'll distribute the geoip db from people.debian.org, like olasd, probably :P 18:21:22 #action tumbleweed to find a solution to the geoip issue 18:21:33 tumbleweed: or from the streaming backend 18:21:40 (just wget it once) 18:21:44 yeah 18:21:46 #action Advice/training for directors (/talkmeisters) 18:21:58 who's advice/training? :P 18:21:59 paddatrapper: didn't you mean to topic that? 18:22:15 wouter: yes, that is the next point on the agenda 18:22:16 (maybe undo this one first...) 18:22:26 #undo 18:22:26 Removing item from minutes: 18:22:29 urg 18:22:34 #topic Advice/training for directors (/talkmeisters) 18:22:38 :) 18:22:39 * paddatrapper needs coffee 18:22:39 hello! 18:22:56 * pollo frowns at the idea at this hour 18:22:58 :P 18:23:06 or sleep :P 18:23:18 at this point either would work 18:23:20 didn't we say we wanted to decide on which recording stack we're going to have first, before thinking about training? 18:23:29 we did do some work on this after the last meeting 18:23:40 and since we're not quite decided yet... 18:23:44 oh, okay 18:23:52 #link https://storm.debian.net/shared/iVvMB3RMXFtTen_jbtZ_FIh0-Ob54DaYvXvxaZx7_Ls 18:24:18 pollo: what still needs doing on it? 18:24:27 I assume technology specific stuff 18:24:41 #undo 18:24:41 Removing item from minutes: 18:24:46 that link is for presenters 18:24:58 tumbleweed: do you have the directors, etc training link? 18:25:34 ah, I confused the two then, I don't think we did work on that one yet for the reasons wouter mentionned 18:25:40 ok cool 18:25:54 I don't think we had a pad for training yet 18:26:09 #agreed will only start working on training for directors once video stack is finalized 18:26:11 offtopic, but: Issues created: https://salsa.debian.org/debconf-team/public/data/dc20-online/-/issues 18:26:17 \o/ 18:26:22 #topic Advice/training for presenters 18:26:27 (and not offtopic) 18:26:33 #link https://storm.debian.net/shared/iVvMB3RMXFtTen_jbtZ_FIh0-Ob54DaYvXvxaZx7_Ls 18:27:13 does anything still need to be added there? 18:27:16 I think that looks like a good idea 18:27:22 we should start putting this in LaTeX and push it somewhere on salsa 18:27:30 maybe limit to just one codec though 18:27:37 ideally the codec we'll be using for live streaming 18:27:42 nattie: I seem to recall you and I offering to edit and compile that? 18:27:49 wouter: don't think the codec really matters 18:27:49 or pandoc +md -> LaTeX 18:27:56 paddatrapper: yeah, i think so 18:27:56 sphinx, like our other docs 18:28:04 wouter: it'll be going through voctomix 18:28:04 +1 for sphinx 18:28:07 expecting vocto to transcode VP9 -> mp4 at live speeds might be expecting too much? 18:28:08 ah, that's not a bad idead :0 18:28:24 wouter: that's not how vocto works 18:28:27 or am I just being a pessimist here? 18:28:33 the ingest will decode to raw video, and give that to vocto 18:28:40 oh, okay 18:28:42 the recording will encode to something 18:28:45 the stream will encode to something 18:28:48 #action paddatrapper to edit and compile presenter advice to video-team docs 18:28:56 #action nattie to edit and compile presenter advice to video-team docs 18:28:58 so we run the ingest on a different VM from the voctocore one then? 18:29:13 maybe? maybe note 18:29:22 the ingest is almost free, it's just decoding video 18:29:41 and bandwdith into vocto 18:29:45 we should probably test and see 18:29:45 yeah, okay, I just thought vocto would be transcoding, but if that's not the case, no worries then 18:29:47 probably makes sense to do it on the core VM for that reason 18:29:50 and we have 10Gbit there, so yeah 18:30:19 #topic Any other business 18:30:32 so, yeah 18:30:44 zigo said that there is the possibility for block storage 18:30:47 do we want to use that? 18:30:49 it'll certainly be easier to have voctoweb run ingest shell commands if it's on the same host as the voctocore 18:31:02 going back on the video stack topic: we need to make sure the core vms can be allocated enough disk space for 1/ having the pre-recorded videos stored; 2/ doing the voctocore recordings 18:31:06 I can make SReview use block storage semi-directly, which could speed things up slightly 18:31:12 wouter: vocto (like what is in the voc repo/package" is like an audio mixer - it has electroncic and a UI, but no mics or speaker hardware. all that needs to be plugged in 18:31:29 #info there is the possiblity for block storage for the infomaniak VMs 18:31:31 CarlFK: right 18:31:58 essentially, I'm querying whether we want to run SReview on vittoria or at infomaniak 18:32:05 and if the latter, whether we use block storage or NFS 18:32:20 #agreed we need to make sure the core vms can be allocated enough disk space for 1/ having the pre-recorded videos stored; 2/ doing the voctocore recordings 18:32:21 I should note that I haven't used the block in anger yet, so 18:32:35 block storage* 18:32:42 what do people think? 18:32:56 * tumbleweed sees what storage options infomaniak has 18:33:04 I don't know how to record to "block storage", whatever that is 18:33:12 can anything else use block storage directly? (aside from sreview) 18:33:13 so we'll have to do a file transfer either way 18:33:27 olasd: block storage is essentially Amazon S3 and lookalikes 18:33:29 they'll go up to 2TB 18:33:42 wouter: that sounds overly complex for little gain 18:33:49 wouter: yeah, I don't know how to record to that; I don't think ffmpeg has a s3 output, nor gstreamer? 18:34:09 if we can get vms with enough disk space, that should be easier 18:34:17 and 2TB should be enough 18:34:19 you can mount that as a filesystem (s3fs) 18:34:22 ivodd: I don't think it is complex, but meh 18:34:23 just sayin 18:34:39 DLange: sreview is quite I/O heavy 18:34:41 wouter: you wanted an opinion from someone else... 18:34:51 also, modern SReview has sreview-copy which transparently copies into S3 if wanted 18:34:55 DLange: I wouldn't want to chance live recordings on a fuse backend 18:34:59 ivodd: sure 18:35:03 pollo: can you point me to a wiki where I can make notes about playing videos? (I hope I didn't miss this last week) 18:35:18 I can just tell you it works better than NFS 18:35:24 CarlFK: https://salsa.debian.org/debconf-team/public/data/dc20-online/-/issues/3 18:35:27 so if only these two are your choices... 18:35:32 DLange: that's a low bar to pass 18:35:38 olasd: full ack 18:35:38 tumbleweed: thanks 18:36:03 will content team ask presenters if they want make live streaming or send recorded talk? 18:36:20 phls: I think they'll strongly encourage them to record 18:36:34 okay, was just idly wondering, but I guess if it makes the streaming side of things more complex, it's probably not worth it 18:37:00 wouter: hrm, how does it relate to streaming? 18:37:13 tumbleweed: er, I meant the recording side of things 18:37:15 tumbleweed: s/streaming/recording/ 18:37:17 right 18:37:25 #agreed we won't use block storage for recording, as it makes things more complex 18:37:26 he wants to plug the tape recorder into the line out of a powered speaker :D 18:37:45 CarlFK: well, it is how FOSDEM does things 18:37:51 (the live stream is recorded) 18:37:58 see! 18:37:59 and yes, that's crazy and has shitty downsides 18:38:09 Any other business? 18:38:24 still want to know whether to run sreview on vittoria or at infomaniak 18:38:47 even if we don't do the block storage, it might still be a good idea (or not) to run SReview at infomaniak at one of the VMs there 18:38:53 probably infomaniak would be good? 18:39:00 wouter: when do you want to start setting up sreview? 18:39:03 makes file transfer easy 18:39:14 paddatrapper: hrm? 18:39:14 we'll want to archive the recordings on vittoria, the infomaniak offer is time-limited 18:39:17 tumbleweed: eh, haven't thought of that yet :) 18:39:26 tumbleweed: s/easy/fast 18:39:37 olasd: yeah, sure, but I'm mostly concerned about massive file transfers during the conference 18:40:07 tumbleweed: I could probably start off slowly if the VMs at infomaniak are going to stick around until debconf 18:40:24 zigo didn't want us to consume vast VMs for months 18:40:30 (on a side note, I could also fix the bugs in ansible around SReview that I have been sitting on since dc19...) 18:40:43 so it'd probably make sense to delay creating the big 2TB VM for a while (unless he's OK with that) 18:41:04 I'd think we'd want that one to be an NFS server and nothing else? 18:41:10 I guess all the sreview stuff is ansibled enough that we don't need to be worrying abuot this right now 18:41:21 you'd guess slightly wrongly ;) 18:41:27 heh 18:41:28 2TB is also probably more than we need; are we splicing the original recordings or are we recording our playback? 18:41:31 most of it is ansibled fine, but some of it is slightly buggy 18:42:01 it would be cool if I could work on this a bit more while I have VMs that are going to stay around for more than a debconf 18:42:01 olasd: presumably we're recording our playback, for our sanity? 18:42:20 tumbleweed: I don't know, which is why I'm asking ;) 18:42:20 Also splicing isn't well supported 18:42:56 anyway, nothing further that *needs* discussing now from me 18:43:03 because usually I go "don't have the time to fix this now, I'll do it manually", and then things like https://salsa.debian.org/debconf-video-team/ansible/-/issues/40 happen... 18:43:26 I think we should be recording our playback 18:43:41 #agreed we will record our own playback 18:43:43 it would be nice if you could use the source instead 18:43:46 mostly because SReview assumes an uninterrupted timeline 18:43:52 wouter: they go up to 32 core. would you just want a single VM for sreview then? 18:43:54 that way, the video could be released immediately when the talk ends 18:44:05 ivodd: then you lose Q&A 18:44:08 but that's "nice to have", not a requirement 18:44:11 tumbleweed: wow, yeah, that's more than enough 18:44:17 tumbleweed: I'm talking for talks where there is no live q&a 18:44:31 ivodd: mmm, good point 18:44:32 ivodd: sure, we could publish those directly 18:44:42 probably bypassing sreview entirely 18:44:49 maybe sreview can do it 18:44:58 (if I can find the energy to look at it again) 18:45:01 it shouldn't be too hard to make it bypass the review cycle 18:45:24 wouter: zigo did ask us to try to use as few resources as possible though, as a 32 cores VM means racking a server for them or something 18:45:51 pollo: I'm not going to actually use the cores, but I don't think he's going to kill me if I end up running ansible on it a few times? 18:45:57 I'll double-check with him to be sure 18:46:03 zigo: ^^ ;-) 18:46:06 (there, done ;-P) 18:46:07 yeah, let's have that discussion with him outside this meeting 18:46:24 #action wouter to talk to zigo about VM resource use 18:46:34 #topic Upcoming meetings 18:46:44 can you also #action me about bypassing the review for prerecorded talks? 18:46:50 #undo 18:46:50 Removing item from minutes: 18:46:53 wouter: we don't want to bypass the review 18:47:00 we want it to be (slightly) different 18:47:05 well, that 18:47:10 bypass recording, essentially 18:47:11 #action wouter to work on review for prerecorded talks 18:47:17 thx 18:47:20 #topic Upcoming meetings 18:47:23 Next week Thursday @ 18:00 UTC? 18:47:28 sure 18:47:42 yeah 18:47:45 #agreed Next IRC meeting - Thursday 9 July @ 18:00 UTC 18:47:54 wfm, mostly 18:48:08 #info Online sprint is 10-12 July 18:48:24 it's a long weekend here, this weekend ('murica day, basically) 18:48:25 according to olasd, also 13/14 ;) 18:48:29 so I'll be hacking 18:48:38 ivodd: t-t-t 18:48:44 not sure yet whether I'll have a lot of time, but I can see 18:48:49 I said /overflow/ 18:49:08 #info can tentatively aim for some video test runs on the weekend of 17-19 July 18:49:25 #endmeeting