12:00:40 <hiro> #startmeeting network-health 2024-10-07
12:00:40 <MeetBot> Meeting started Mon Oct  7 12:00:40 2024 UTC.  The chair is hiro. Information about MeetBot at http://wiki.debian.org/MeetBot.
12:00:40 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
12:00:46 <ggus> it's okay
12:00:52 <hiro> the pad: https://pad.riseup.net/p/tor-nethealthteam-2024-keep
12:01:20 <hiro> ok let us known.. we were unsure between 12 and 1400 utc... so we have margin
12:03:05 <hiro> ok let's start
12:03:21 <hiro> I am fighting collector issues going OOM
12:03:41 <hiro> but please let me know if I need to unblock anyone
12:05:26 <GeKo> not really unblocking
12:05:42 <GeKo> but it's still hard to work with the bw inflation dashboards
12:05:47 <GeKo> due to timeouts i guess
12:05:54 <juga> oh, then maybe it's me blocking?
12:06:04 <juga> ah!
12:06:08 <GeKo> adding resources to the machine has been helping a bit
12:06:32 <GeKo> but it still feels like a lottery when loading those dashboards :)
12:06:48 <GeKo> juga: dunno.
12:07:03 <juga> yeah... thinking how to solve that at least for now
12:07:11 <hiro> juga I think we need to start moving that processing into a script and out of sql queries
12:07:29 <hiro> and create aggregated tables with the data you need
12:08:00 <juga> yeah... metrics-sql-tables#37
12:08:12 <juga> i guess i'll prioritize this week that
12:08:42 <juga> though apart of how the tables/views are created
12:08:51 <hiro> yeah for example you could already use  tor_fusion which does onionperf processing and runs once a day (I think)
12:08:53 <GeKo> i am not sure how that works then but i do like to zoom into different time intervals
12:08:54 <juga> would awesome if we can limit the cpu to the parser
12:09:14 <GeKo> maybe that means like hard-coding 1 week, 2weeks etc. as for the cdf panes
12:09:59 <juga> hiro: is it possible to limit cpu used by the parser?
12:10:09 <GeKo> sometimes it's useful to have custom timeframes, though
12:10:10 <juga> so that there's cpu for queries
12:10:15 <hiro> @GeKo: we could store smaller time intervals and do the zoom out when needed
12:10:24 <GeKo> yeah
12:10:25 <juga> i mean, just for reading
12:10:50 <hiro> @juga the time consuming part in the parser are the materialized views
12:11:01 <juga> cpu, not time
12:11:03 <GeKo> for the cdf i am fine with those hard-coded values
12:11:15 <juga> parser could take longer, but not occupy all cpu
12:11:24 <GeKo> but i am not sure whether we want to be a bit more flexible for inflation hunting...
12:11:28 <hiro> if I limit resources then everything will be slower because it runs every hour and won't finish within the hour
12:11:44 <juga> anyway, as commmented in metrics-sql-tables#37, i'd revert refresh from descriptorParser
12:11:53 <GeKo> the classic onionoo problem :)
12:12:01 <juga> and then descriptor will be faster
12:12:13 <hiro> yeah we can revert refreshing
12:12:13 <GeKo> we need to finish before the hour!11!!
12:12:26 <hiro> yeah lol
12:12:52 <juga> ok, i'll do that then right after this meeting
12:13:15 <hiro> but why don't we do the aggregation in tor_fusion? we have the db setup there
12:13:19 <hiro> and it runs once a day
12:13:32 <juga> hiro: after i do that, what if parser is taking all cpu even if it's only for 15min?
12:13:53 <juga> hiro: yeah, i'm going to do aggregation apart
12:13:59 <juga> well, not just aggregation
12:14:03 <hiro> uhm
12:14:20 <juga> but when the data isn't related to tor_fusion
12:14:32 <nipaton> hey todays not thursday! I missed the meeting
12:14:36 <juga> i don't think i should add more code there
12:14:46 <hiro> ok what if I make a jar that doesn't refresh the materialized view and we try and see?
12:14:52 <juga> also, right now would be python
12:15:07 <hiro> ok then you can make your own script if you prefer
12:15:14 <juga> i'd avoid messing up with descriptor or java at all
12:15:23 <hiro> tor_fusion is rust
12:15:26 <juga> yeah, i think that would be better
12:15:38 <juga> yeah, that's what i say in a different script
12:15:46 <hiro> ok sounds good
12:15:52 <juga> ok
12:16:05 <GeKo> nipaton: you did not, it's a different one
12:16:11 <GeKo> this is the network-health meeting
12:16:22 <GeKo> you are welcome here as well, of course :)
12:16:31 <nipaton> haha
12:16:52 <nipaton> does every topic have a schedule?
12:16:54 <hiro> was it anti censorship you were looking for?
12:17:06 <nipaton> yeah I was in the revious one
12:17:17 <GeKo> yeah we have a schedule
12:17:32 <hiro> yeah we are organized in teams
12:17:36 <GeKo> but i am not sure where the place is where one can see all the meeting times at a glance
12:18:07 <hiro> I think maybe the anti-censorship team has something in their wiki on gitlab but I am not sure
12:18:09 <juga> hiro: re. which script to use, look metrics-sql-tables#37 if you haven't yet, maybe i need to explain it better
12:18:30 <nipaton> hmm, that would be helpful
12:18:38 <hiro> ok juga will continue talking on the issue
12:18:46 <juga> ok
12:18:57 <GeKo> nipaton: https://gitlab.torproject.org/tpo/anti-censorship/team
12:19:04 <nipaton> kudos
12:20:01 <hiro> I think we do not need to do everything t hat you mention on the ticket all at once
12:20:14 <hiro> I think for now we can just add tables to the db for the data we need
12:20:24 <juga> #37, sure, step by step
12:20:51 <hiro> anyways
12:20:53 <hiro> @sarthikg: how is it going for you? are you settling ok?
12:21:57 <sarthikg> going good, finally getting a good hold of tagtor, and that eventually smoothens the way to metrics 2.0
12:22:05 <hiro> \o/
12:22:26 <GeKo> nice!
12:23:40 <hiro> nice nice!
12:23:59 <hiro> if you have any question please reach out to any of us
12:24:30 <sarthikg> sure sure, will definitely do!
12:24:46 <hiro> @ggus is everything ok on your side?
12:25:50 <ggus> we need to think on the next meetup agenda
12:26:26 <ggus> do you prefer to discuss after the meeting?
12:27:36 <hiro> yes let's think about it, I  think also possibly thinking about 38c3 meetup and activities
12:28:16 <ggus> oh wow, it's already the end of the year!
12:28:22 <hiro> yep
12:28:29 <ggus> sounds good
12:28:36 <hiro> and cfp is either opening or closing soon
12:28:54 <ggus> https://gitlab.torproject.org/tpo/community/relays/-/issues/100
12:30:44 <hiro> I think for the meetup we should figure out how many contributions we want from the community (like 1 or 2? I'd do 1, but maybe we could have 2) and then think about the nh update given everything that happened
12:32:26 <hiro> oook if everyone is groot we can end the meeting
12:32:41 <GeKo> i am
12:32:44 <juga> me too
12:33:00 <ggus> i'm good too
12:33:11 <hiro> oook!
12:33:15 <hiro> #endmeeting