17:59:18 #startmeeting 17:59:18 Meeting started Tue Jun 16 17:59:18 2020 UTC. The chair is paddatrapper. Information about MeetBot at http://wiki.debian.org/MeetBot. 17:59:18 Useful Commands: #action #agreed #help #info #idea #link #topic. 17:59:27 Agenda: http://deb.li/oNCD 17:59:32 0/ 17:59:34 o/ 17:59:37 \o 17:59:43 hi 18:00:42 Heyy o/ 18:00:50 #topic Online Minidebconf Review 18:01:02 o/ 18:01:11 My notes of the miniconf - https://storm.debian.net/shared/GocW9F49JKgZqxwP0elMicqbHm-8L3RuAmoDOM8YLny 18:02:08 how was the load on the jitsi/jibri instance? 18:02:13 network and cpu 18:02:37 The main resource hog was Jibri, highvoltage should have the numbers though 18:03:10 Jitsi was faily light, even when we have ~20 people in the room 18:03:18 * nattie is here but half occupied 18:03:28 hi 18:05:31 I think we have covered most of the technical things before and I'm sure we will encounter them again in the next topic 18:06:04 Are there any other things to note that aren't in the storm pad? 18:07:11 I wonder if we colud build more lua plugins to make it do what we want 18:07:31 for prosody? 18:07:44 depends what we need it to do I guess 18:08:17 paddatrapper: yeah 18:08:20 err pollo 18:08:57 that said, I don't know what role prosody plays in jitsi meet 18:09:03 the main thing I would like is individual participants feeds into vocto, but that's not prosody, that jitsi-videobride (SFU) IIRC 18:09:31 IIUC prosody does auth + chat 18:09:35 yeah, that would be videobridge 18:09:50 also, that's later in the agenda 18:10:01 moving on then? 18:10:02 hello o/ sorry have a headache so I'll catch up / reply tomorrow 18:10:22 #topic DC20 Online 18:10:38 Tracking progress 18:11:03 yea to all ? 18:11:16 +1 18:11:18 probably 18:11:34 could you c/p items from the agenda so we know what we're talking about? 18:11:45 shall we create a dc20-online salsa repo for this? 18:11:46 Regular IRC meetings 18:11:50 Online sprint 18:11:53 Test runs 18:11:58 Using salsa issues 18:12:00 I don't think we'll manage if we don't meet regularly, have clear objectives and test shit waaay in advance 18:12:10 pollo++ 18:12:15 so, weekly meetings from now on? 18:12:23 wfm 18:12:27 +1 18:12:43 tumbleweed: in the debconf-video-group or elsewhere? 18:12:45 Sounds good, though can we shift to Thursdays? 18:12:58 (though this is my first meeting here) 18:13:06 pollo: I was thinking debconf-data 18:13:15 having it farther from the debconf team meeting sounds good in general 18:13:19 wfm 18:13:27 +1 18:13:28 tumbleweed: if we're not putting scripts and shit there that works 18:13:48 pollo: I mean, for the salsa issues bit 18:13:49 I'll put scripts and shit wherever I please :P 18:14:08 (but ideally in ansible) 18:14:15 (although maybe dc20 will go to fast for that) 18:14:18 #agreed regular meetings on Thursdays at 18:00 UTC 18:14:40 jitsi + ansible is a real PITA 18:14:52 jitsi + cfg management in general 18:15:01 Created: https://salsa.debian.org/debconf-team/public/data/dc20-online 18:15:13 Yeah we're still working on getting the debian social jitsi instance in config management 18:15:21 got an idea of a timeframe for the sprint? 18:15:28 #agreed track issues here - https://salsa.debian.org/debconf-team/public/data/dc20-online 18:15:35 (and granted maintainer rights to the video team) 18:15:53 olasd: I'm available any time after the 1st week of July 18:16:07 I was about to suggest early july 18:16:22 which leaves us time to poke at things before we get a more focused sprint done? 18:16:43 paddatrapper: is that after the 29th-5th ot 6th to 12th ? 18:16:49 (or to not poke at things and rush through a more focused sprint) 18:16:50 State of the Map is 4th and 5th July. Once that is over I'm freed up a lot 18:16:52 (ahem) 18:17:23 I should know on Wednesday how similar a stack we will be using for that 18:17:36 I suspect your feedback from SOTM will also be useful, whatever the stack you're using will be 18:17:39 anyway, that time frame wfm 18:18:03 #agreed online sprint during early July 18:18:04 when does cfp end? 18:18:35 https://debconf20.debconf.org/schedule/important-dates/ 18:18:39 thanks 18:18:44 end of july 18:19:01 err 18:19:01 #info CfP end 31 July 18:19:02 feels late 18:19:04 that seems to be wrong 18:19:05 https://debconf20.debconf.org/cfp/ 18:19:09 Submission deadline: Sunday, July 5th Acceptance notification: Monday, July 20th 18:19:13 #undo 18:19:13 Removing item from minutes: 18:19:18 yeah that sounds more like it 18:19:23 #info CfP end 5 July 18:19:33 one month to have people submit a pre-recording sounds better 18:19:40 (submit and check and ...) 18:19:53 Content format? 18:20:07 sorry, last point in this sub-item: test runs 18:20:38 Definitely a good idea 18:20:55 Could do a microconf testing the stack 18:21:00 I think that could be the goal of the sprint: by the end of things, being able to do a test-run for a full track 18:21:03 or just demo talks 18:21:14 well, for a full block of talks + bofs 18:21:33 Yeah, and then expand after the sprint to multiple tracks 18:21:46 if there's enough content to have to do that 18:21:53 (which we'll know by the time of the sprint) 18:22:14 true, that we shall 18:22:23 #agreed goal of the sprint: by the end of things, being able to do a test-run for a full track 18:22:35 s/full track/block of talks :p/ 18:22:43 #undo 18:22:43 Removing item from minutes: 18:22:54 #agreed goal of the sprint: by the end of things, being able to do a test-run for a block of talks 18:22:55 :) 18:23:02 Content format 18:23:26 (could do with some #topics) 18:23:40 #topic DC20 Online - Content Format 18:24:07 * terceiro tunes in 18:24:18 What content formats do we want to support? Prerecorded talks, live presentations, live bof-style discussion with multiple 'main' participants/presenters 18:24:27 and how do we do questions 18:24:42 so, I think there's two main types of content we'll have: talks and BoFs. 18:25:22 there are also the in-between panel style events. e.g. meet the tech-ctte 18:25:30 (lots of questions, but main presenters) 18:25:37 right 18:26:11 I think for regular talks, we should strongly recommend folks to prerecord and do a live QA 18:26:14 for talks, my opinion is that the best quality for attendees (and for posterity) would be provided by a pre-recording 18:26:34 +1 18:26:48 (even though that's more logistics for presenters) 18:27:01 While the live was nice during MDCO, there were lots of issues there 18:27:18 (and, arguably, for us; people will need to watch all this content beforehand) 18:27:35 Live questions would be good 18:27:49 TBH every other online event is just live. if things happen, they happen 18:28:01 I agree that pre-recording gets better quality 18:28:04 I feel the best way for us to enforce this would have folks that want to do a live talk ask us first if they can 18:28:11 but it's more upfront work for everyone 18:28:14 terceiro: the only online event that I've seen go fully live was MDCO 18:28:18 otherwise I think a large majority will go with live 18:28:34 olasd: really? my experience if the opposite 18:28:37 pollo: +1 18:29:08 we can vet with them if they have a stable network connection, good audio + video, etc. and then say yes 18:29:13 all academic conferences I've had friends present at asked for pre-recordings; I've seen some infosec conferences do that too 18:29:48 most LGM events were pre-recorded too 18:30:44 so for talks we strongly recommend recordings 18:30:48 I think for talks, the QA part can be either live or IRC, I don't really mind 18:31:06 live as in "presenter is live, folks ask questions on IRC" 18:31:26 #agreed regular talks should be pre-recorded and people can request to do it live if they really want to 18:31:27 written questions in IRC seems more manageable, yes 18:31:31 I don't think we want to have to deal with people asking questions on Jitsi 18:31:32 we should set a deadline for receiving the recording, so we get them more than 5 minutes before the talk 18:32:11 but bofs/panels/etc need to be live by definitions - isn't it easier to have just 1 way of doing things? 18:32:16 ivodd: +1 probably before the start of the conference? 18:32:22 #agreed questions in IRC with the presenter answering live 18:32:33 terceiro: the one way we had of doing things was really not great 18:32:35 tumbleweed: we don't have to set that deadline now, we just have to set it before the talks are approved 18:33:01 tumbleweed, ivodd: my gut says at least 15 days after acceptance notification 18:33:07 agreed, we want to check the videos to see if everything is OK first :) 18:33:25 well I'm busy hacking on important-dates right now, so I'll do that 18:33:53 August 16th? (one week before the conference starts) 18:34:02 terceiro: bofs by definition are kinda janky anyway, I don't mind that much if they go haywire 18:34:16 I feel the video team is signing up for a lot of upfront work, but I'll shut up 18:34:27 pannels we can take time with presenters in advance to make sure everything is ok 18:34:29 #agreed recordings are needed on August 16 18:34:54 terceiro: imo it's the same stack, but we press play in ffmpeg for pre-recorded talks 18:34:55 terceiro: playing a recording feels like something that *should* be simple :P 18:35:00 tumbleweed: lol 18:35:13 #agreed BoFs and panels happen live and issues are dealt with as they come up 18:35:17 tumbleweed: famous last words? 18:35:24 yes but pre-reviewing N recordings is not 18:35:54 oh, we can call upon folks that review talks for that! 18:36:04 play it 10x or something 18:36:27 nattie: this is where your speaker support team comes in, right? 18:36:51 one thing is certain, we need a "how to record a video" guide 18:36:51 I'd expect video team just reviews for technical details, and probably doesn't play through every talk 18:37:28 I hope to be wrong 18:37:47 I think we'll need an android and a Debian version too 18:38:01 I would expect a number of our speakers would be lazy and just wing it with live presentations 18:38:50 I guess we can move on 18:39:01 some tutorial for people record thei talks? 18:39:01 Probably can get the speaker support team to develop a how to guide 18:39:10 that's later on the agenda 18:39:20 *nod* 18:39:39 #topic DC20 Online - Streaming 18:40:02 I strongly suggest we put Vocto or something similar between Jitsi and the stream 18:40:14 it also allows us to play recorded talks easily 18:40:18 +1 18:40:25 we'll need to deal with the latency issue though 18:40:28 +1000 18:40:44 How much latency does it add? Has someone tested it? 18:40:46 VNC and telnet are also options there 18:40:51 pollo did some testing 18:41:09 well, do we? we don't have to have previews for everything 18:41:11 I tested how feasible it was to run voctogui locally and voctocore remote 18:41:11 ah latency between VoctoGUI and core 18:41:23 not a great idea 18:41:25 we could have a few presets and buttons to switch between them 18:41:50 olasd: how would the director see what happens? 18:41:51 if it's only for switching between a recording and a jitsi room, it's probably not a big issue 18:42:05 on the stream, we can expect ~30s delay 18:42:12 if you want to switch between multiple streams (from multiple presenters), that's a different issue 18:42:18 (but we can't do that now anyway) 18:42:25 pollo: ok that's about the same as Jitsi streaming anyway 18:42:42 paddatrapper: that's just ballpark numbers from previous DC 18:42:51 ah 18:42:52 I didn't test anything streaming related 18:43:34 we can get a lower latency rtmp stream out of the nginx though, right? 18:43:39 yes 18:43:57 #agreed have something (vocto probably) between Jitsi and the stream 18:44:29 If we encourage people to ask questions throughout the talk and collate, then the latency isn't as much of an issue 18:44:35 if we get a decent stream latency, I guess we can manage voctocore in a terminal without a UI 18:44:41 Though it doesn't help for followup questions 18:44:45 pollo: to do simple switching between jibri, loop and recording, the director doesn't need full live previews 18:44:51 tumbleweed: that's the next item on the agenda: 'Does anyone want to try to setup an alternative streaming stack (in addition to the one we have) with lower latency' 18:44:56 only three buttons and maybe a still for each 18:45:05 (updated every second or 5) 18:45:22 ivodd: I wasn't thinking of that as an alternative stack 18:45:28 olasd: would that mean writing a client? 18:45:30 but rather that the video director doesn't need to be behind the 30 delay 18:45:33 pollo: yes 18:46:06 3 CLI commands sounds simpler, but if people want to do that I'm not against it :) 18:46:10 tumbleweed: right, but if it works for the director, how much more work would it be to make it work for others as well? 18:46:35 yeah, we can start with cli commands 18:46:58 I've played with that before, action me for it plz 18:47:12 btw, fosdem has a voctomix web UI 18:47:21 hmm, interesting 18:47:23 #action pollo to develop CLI commands to work with voctocore 18:47:23 well, a skeleton of a voctomix web ui 18:47:31 that is very interesting 18:47:45 can we get an #info? 18:47:46 #info FOSDEM has a skelton votomix web UI 18:47:49 I'll also check that 18:47:59 #link https://github.com/krokodilerian/vocto-web-mix/ 18:48:05 #chair pollo 18:48:05 Current chairs: paddatrapper pollo 18:48:09 :P 18:48:24 but it's really what I was thinking of: 3 buttons, and some stills for sources refreshed every second or so 18:48:28 #action pollo to check the FOSDEM vocto web client too 18:49:01 #topic DC20 Online - Video Stack 18:49:10 olasd: would you have time to try and test stream latency? 18:49:24 to see how low we can get on the geolocated frontends 18:50:01 paddatrapper: you skipped the latency subpoint 18:50:15 #topic DC20 Online - Streaming 18:50:17 So I did 18:50:18 and, yes, I've been meaning to try something along those lines 18:50:26 #action olasd to try reduce stream latency 18:50:32 neato 18:51:07 (thanks :D) 18:51:51 Does anyone want to try to setup an alternative streaming stack (in addition to the one we have) with lower latency? 18:52:09 paddatrapper: using something else than nginx rtmp ? 18:52:26 I think it's not going to be an alternative stack, just different settings for the existing stack 18:52:26 pollo: Inferring from the agenda point - yes 18:52:39 at least if I'm working on it I'll try that first 18:52:53 yeah I think that is a good idea 18:52:54 (nginx-rtmp-module can just push to other rtmp endpoints) 18:53:25 bouncing between endpoints increases the latency though 18:53:32 sure 18:53:49 can't really go faster than light 18:53:57 and having everyone connecting to the central nginx increases round-trip latency and load on the central nginx 18:54:01 as said before, it that helps I wouldn't mind it if we use some commercial service as an endnote 18:54:10 but we can clearly do better than 60 seconds 18:54:22 let's stream to youtube! job done :P 18:54:26 heh 18:55:04 I think our primary issue is that video players really, really like lots of buffering before playing 18:55:34 but, anyway, I've been actioned, we can move on 18:55:36 #topic DC20 Online - Video Stack 18:55:43 We used Jitsi + Jibri for the miniconf. It has its limitations (see above), but we know it's usable. Does someone want to set up a POC for another option? 18:56:04 BillowConf won't be ready in time 18:56:21 would BBB be an acceptable option? 18:56:28 Unless people are wanting to help with code 18:56:35 olasd: I checked, and RTMP support is bad 18:56:48 It is also a dog show to setup and maintain 18:57:00 isn't jitsi too? 18:57:08 a tad less 18:57:21 tumbleweed: at least we can run Jitsi on Debian 18:57:21 AFAIU BBB only runs on a particular version of Ubuntu 18:57:27 16.04 to be exact 18:57:28 do we care? 18:57:51 Their installer was also broken last time highvoltage tried to install it 18:58:02 I know BBB scales better for large rooms 18:58:16 but I don't have any first-hand sysadmin experience with it 18:58:40 Both highvoltage and I tried to install it and gave up 18:58:48 ok 18:59:26 then I guess we should focus on live interaction going through jitsi/jibri the best we can 18:59:42 sounds good 18:59:49 paddatrapper: if we use vocto, do we need jibri to do streaming? 18:59:52 #agreed we should focus on live interaction going through jitsi/jibri the best we can 19:00:06 pollo: yes, it is what gets Jitsi to RTMP 19:00:37 essentially it captures the chrome window using FFmpeg X11 capture 19:00:51 (ew) 19:00:57 very 19:01:02 you don't say 19:01:56 but it's jitsi -> jibri -> vocto -> nginx -> rtmp right ? Not jitsi -> jibri -> rtmp -> vocto ... 19:02:17 jibri just shells out to ffmpeg 19:02:24 we can pipe that to whatever 19:02:25 yeah ok good 19:02:28 Jibri acts as a Jitsi client 19:03:01 being virtual this year I don't know if there will be much I can do to help, but I will be available, happy to be tasked... 19:03:05 that means we don't have to care about "Jibri can either record or stream, not both." 19:03:24 turns out it can with enough Jibri instances 19:03:38 but yeah, I suggest we record higher up the chain - vocto probably 19:03:41 yeah, we should record on vocto 19:03:43 +1 19:03:54 We seem to have settled on voctomix for real-time mixing. Any other suggestions? 19:04:14 I think it's a decent core; without the gui, we shouldn't have much issues with it 19:04:19 *ahem* 19:04:23 RattusRattus: you can most definitly help with reviewing pre-recorded talks and to write a guide on how to have good audio at home :) 19:04:55 are we using voc2mix or the debian buster package? 19:05:05 haha 19:05:34 * pollo hasn't bee in the loop for ~1 year... 19:05:57 what's voc2mix? 19:06:02 tumbleweed: voctomix 2 19:06:19 let's say going to voc2mix overnight at CCC was... quite an experience 19:06:39 ok, so not ready for prime time then 19:06:40 "oops we messed up the old voctomix setup, we can't go back now" 19:06:54 "oh btw if you click this button the core segfaults" 19:07:07 ok will write gude over next week or so (can also cover webcam / lighting). yep can review as well. 19:07:14 olasd: ouch! 19:07:16 RattusRattus: \o/ that'd be great 19:07:17 :p 19:07:17 #agreed use vocto-core 19:07:35 for jitsi, do we want 1 central servers or multiple ones? 19:07:38 #action RattusRattus to write up a guide to pre-recording for presenters 19:07:53 RattusRattus: shout if you want proof-reading 19:07:59 If we have a bunch of regional mini-confs tracks, 400ms latency to jitsi might be bad 19:08:01 maybe they debugged it more since congress; but I don't think they had much live events to do so since then 19:08:27 paddatrapper: ta! 19:08:55 also, do we want jitsi/jibri/vocto on the same server, or vocto in a separate VM? 19:09:19 if it's all on the same hypervisor I guess it would be ok 19:09:26 Jibri will need to be on other instances if we want more than one jibri instance 19:09:55 Running them all on the same hypervisor as Jitsi in their own VMs causes WebRTC issues... 19:10:11 we just need to make sure the jibri -> vocto network is good then 19:10:30 all of the core video stuff should have fast network to each other 19:10:35 if they both run in the same data centre we should be good 19:10:56 #info all of the core video stuff should have fast network to each other 19:11:30 where do we play pre-recorded talks from? 19:11:39 do we skip jitsi/jibri altogether? 19:11:48 yeah, definitely 19:11:50 that would make sense to me 19:12:01 feed it in as a vocto source 19:12:04 the framerate would make things difficult in Jitsi 19:12:26 #agreed for pre-recorded talks, we'll feed them to vocto directly and skip jitsi 19:12:50 progress! 19:12:58 #topic DC20 Online - Advice for presenters 19:12:59 do we have prefered resolution / frame rate / encoding etc for pre-recorded talks? 19:13:23 hello! 19:13:32 16:9, 720p? 19:13:35 * RattusRattus can then add to the 'how to' 19:13:43 * RattusRattus waves to nattie 19:13:44 25Hz? 19:13:49 25/30 framerate 19:13:51 tumbleweed: I worry about lights 19:14:07 we probably need to jig the stream for a particular framerate, though 19:14:08 pollo: anything we choose will have issues with lights somewhere in the world 19:14:14 "use a framerate that matches your local power grid" 19:14:25 pre-record I will ask for nateral light / give advice on direction / shaddows etc 19:14:31 olasd: ACK 19:14:41 olasd: Five frames per second should be enough for anybody. 19:14:45 #info use a framerate that matches your local power grid 19:14:47 I guess anything recorded in front of a webcam, won't get visible artifacts from 25->30 or vice-versa 19:15:02 I assume we'll transcode everything if needed for pre-recorded 19:15:06 some webcams will only record at 30, too 19:15:08 120fps slow-mo cooking with pollo 19:15:21 gwolf: sounds good to me as you should mostly show your slides anyway 19:15:33 (winkwink) 19:15:35 our publishish recordings will be the output of the vocto stack, so they will be transcoded in the process 19:15:36 RattusRattus: yeah, makes sense 19:15:43 *published 19:16:20 tumbleweed: ack - was talking about pre-recorded talks comming in to us... 19:16:57 yeah, I think we do whatever we have to do to ingest 19:17:11 ack 19:17:17 our "play" script can detect framerate and send the right ingest to vocto 19:17:19 ...alternatively, a brightness adjustment filter could be added to jibri to account for line power/webcam settings being off 19:17:27 (jibri is... just too dirty!) 19:17:33 * gwolf is sorry, I'm just here making noise 19:17:38 and I should be cooking lunch! 19:17:40 o/ ! 19:18:19 why does line power matter at all if you use LED light bulbs? 19:18:21 I think we should transcode/filter the incoming pre-recorded talks anyway to make them work with what we use 19:18:53 p2-mate: cos LEDs flicker at line rate for MOST 'bulbs' 19:19:02 * paddatrapper looks at the incandencents surrounding me 19:19:05 p2-mate: and because lots of people (like I) don't : 19:19:29 p2-mate: cheep bulbs are LED chain and if you are lucky a ballast resistor 19:20:00 even those IKEA tradfri dimmable things? 19:20:17 I'm sure we can discuss the specifics of electronic bulbs after the meeting ;) 19:20:26 p2-mate: no idea never taken one apart. anyway we are way off topic 19:20:50 We need to determine minimum specs for doing live talks 19:20:52 paddatrapper: I don't think we should be doing *too much* work on editing incoming video 19:21:10 I doubt we have the capacity 19:21:10 tumbleweed: not editing, but making sure everything isn't yellow and flickering 19:21:22 paddatrapper: I think we'll need hard data for that, as it depends on how many people are on Jitsi 19:21:43 Is someone willing to do the testing? 19:21:51 fwiu, it's 2mbps/person on jitsi 19:22:18 pollo: is that to the Jitsi server? 19:22:23 Or to each participant? 19:22:26 on the client, quality is configurable, though 19:22:40 paddatrapper: to participants 19:23:18 #info it's 2mbps/person on jitsi to participants, but quality configurable 19:23:18 tumbleweed: yes, but ideally we don't want to rely on that 19:23:49 I suspect CPU grunt is the bigger constraint than bandwidth 19:23:58 (or our ancinent-thinkpad audience) 19:24:00 * RattusRattus can test stuff if it is already setup (i.e. ill be your bitch) 19:24:02 That is what we found during MDCO 19:24:18 #action paddatrapper to gather info on CPU usage during SotM 19:25:14 I think we can end on this? 19:25:17 RattusRattus: jitsi instance is set up, we just need people to throw at it 19:25:34 and we should get a voip gateway to it up 19:25:47 And test their machines' resource usage 19:25:52 tumbleweed: yes definitely 19:26:08 the sponsorship team is looking into that? 19:26:08