12:02:09 <hiro> #startmeeting network-health 2024-10-14
12:02:09 <MeetBot> Meeting started Mon Oct 14 12:02:09 2024 UTC.  The chair is hiro. Information about MeetBot at http://wiki.debian.org/MeetBot.
12:02:09 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
12:02:22 <hiro> pad: https://pad.riseup.net/p/tor-nethealthteam-2024-keep
12:02:27 <GeKo> hi!
12:03:45 <juga> o/
12:05:40 <GeKo> hiro: what do you think about https://gitlab.torproject.org/tpo/network-health/metrics/metrics-sql-tables/-/issues/41 ?
12:05:41 <hiro> oook I do not have much for today.. I am big into the parser bugs xD
12:06:19 <hiro> yep I can rename that
12:06:27 <hiro> I could do that  with the stale statuses change
12:06:43 <GeKo> okay, sounds good
12:07:59 <hiro> is there something else I should pick?
12:08:29 <GeKo> i did not mean to put it on your plate, fwiw
12:08:35 <hiro> I really hope  to get to the point where we can have a fresh database and start loading the data xD
12:08:38 <GeKo> i am fine doing the changes myself :)
12:08:57 <hiro> but I might have to change the table f ields in the parser too
12:09:10 <GeKo> yes, i can do that patch as well ;)
12:09:30 <GeKo> what about shortening the read/write lines in the bandwidth table to save space?
12:09:54 <hiro> which ones we should save you think?
12:10:05 <hiro> I wanted to cut out the json
12:10:14 <GeKo> let me find the ticket
12:10:16 <hiro> but I am using that to build the bandwidth tables out of it
12:10:54 <GeKo> https://gitlab.torproject.org/tpo/network-health/metrics/descriptorParser/-/issues/102
12:11:31 <GeKo> it seems to me we waste a bunch of space here
12:12:01 <hiro> so the parsed json comes handy for the extra_info_bandwidth_history table
12:12:37 <hiro> and I do not think we use the original string anywhere
12:13:00 <hiro> @juga do you use it somewhere?
12:13:17 <juga> i did use some of those values
12:13:26 <juga> but with the last changes, i don't need to
12:13:36 * juga checking what used before
12:14:29 <juga> i think i only used write_history and read_history
12:14:43 <juga> anyway, i'm changing that to use the new table
12:15:07 <GeKo> hrm, could we recreate all the desc info out of the json blob?
12:15:36 <GeKo> i guess you can infer the period from subtracting two consecutive timestamps
12:15:42 <hiro> yeah
12:15:42 <GeKo> so that's fine
12:15:57 <juga> what i was never using is "line", that's not even json, just text
12:16:03 <GeKo> and you can translate the first timestamp to the date the measurements started
12:16:08 <hiro> yes
12:16:38 <GeKo> so, we should be good just with the json then
12:16:58 <hiro> another possibility is leave the original line and  do the processing later
12:17:16 <hiro> it is just a matter of moving the  code from the extra_info parser to the status builder
12:17:21 <GeKo> that would be my preferred option but i am not sure how difficult that is
12:17:40 <GeKo> in general i like to see as much unprocessed descriptor data in the db as possible
12:17:47 <GeKo> given that should be our raw source
12:18:04 <GeKo> like the entries of our compressed tarballs right now
12:18:04 <hiro> yeah you are right.. I'll do that
12:18:23 <GeKo> thanks
12:18:26 <juga> for calculating the min over the last 5 values (what there's in a line), i can also use values from an already parsed table
12:18:34 <hiro> we leave the descriptors table less processed and move t he processed  data in other tables
12:18:46 <GeKo> yeah
12:18:55 <GeKo> that would be better imo
12:19:08 <hiro> @juga could you use the data f rom the extra_info_bandwidth table?
12:19:14 <juga> yes, yes
12:19:23 <juga> my plan is that
12:19:30 <juga> still on that though
12:19:31 <hiro> ok let me know if you need the data in a different format
12:19:44 <hiro> I know t his is painful.. but we are figuring out  a lot of things at the same time
12:19:47 <juga> is good as it is in extra_info_bandwidth now
12:20:00 <juga> yes, well, it happens :)
12:20:45 <hiro> oook!
12:22:22 <hiro> any other issue we should discuss?
12:22:33 <GeKo> nothing from my side, thanks
12:22:34 <juga> i'm fine
12:22:41 <hiro> otherwise if everyone is good we can end the meeting
12:22:44 * hiro is groot
12:23:45 <hiro> #endmeeting