12:02:09 #startmeeting network-health 2024-10-14 12:02:09 Meeting started Mon Oct 14 12:02:09 2024 UTC. The chair is hiro. Information about MeetBot at http://wiki.debian.org/MeetBot. 12:02:09 Useful Commands: #action #agreed #help #info #idea #link #topic. 12:02:22 pad: https://pad.riseup.net/p/tor-nethealthteam-2024-keep 12:02:27 hi! 12:03:45 o/ 12:05:40 hiro: what do you think about https://gitlab.torproject.org/tpo/network-health/metrics/metrics-sql-tables/-/issues/41 ? 12:05:41 oook I do not have much for today.. I am big into the parser bugs xD 12:06:19 yep I can rename that 12:06:27 I could do that with the stale statuses change 12:06:43 okay, sounds good 12:07:59 is there something else I should pick? 12:08:29 i did not mean to put it on your plate, fwiw 12:08:35 I really hope to get to the point where we can have a fresh database and start loading the data xD 12:08:38 i am fine doing the changes myself :) 12:08:57 but I might have to change the table f ields in the parser too 12:09:10 yes, i can do that patch as well ;) 12:09:30 what about shortening the read/write lines in the bandwidth table to save space? 12:09:54 which ones we should save you think? 12:10:05 I wanted to cut out the json 12:10:14 let me find the ticket 12:10:16 but I am using that to build the bandwidth tables out of it 12:10:54 https://gitlab.torproject.org/tpo/network-health/metrics/descriptorParser/-/issues/102 12:11:31 it seems to me we waste a bunch of space here 12:12:01 so the parsed json comes handy for the extra_info_bandwidth_history table 12:12:37 and I do not think we use the original string anywhere 12:13:00 @juga do you use it somewhere? 12:13:17 i did use some of those values 12:13:26 but with the last changes, i don't need to 12:13:36 * juga checking what used before 12:14:29 i think i only used write_history and read_history 12:14:43 anyway, i'm changing that to use the new table 12:15:07 hrm, could we recreate all the desc info out of the json blob? 12:15:36 i guess you can infer the period from subtracting two consecutive timestamps 12:15:42 yeah 12:15:42 so that's fine 12:15:57 what i was never using is "line", that's not even json, just text 12:16:03 and you can translate the first timestamp to the date the measurements started 12:16:08 yes 12:16:38 so, we should be good just with the json then 12:16:58 another possibility is leave the original line and do the processing later 12:17:16 it is just a matter of moving the code from the extra_info parser to the status builder 12:17:21 that would be my preferred option but i am not sure how difficult that is 12:17:40 in general i like to see as much unprocessed descriptor data in the db as possible 12:17:47 given that should be our raw source 12:18:04 like the entries of our compressed tarballs right now 12:18:04 yeah you are right.. I'll do that 12:18:23 thanks 12:18:26 for calculating the min over the last 5 values (what there's in a line), i can also use values from an already parsed table 12:18:34 we leave the descriptors table less processed and move t he processed data in other tables 12:18:46 yeah 12:18:55 that would be better imo 12:19:08 @juga could you use the data f rom the extra_info_bandwidth table? 12:19:14 yes, yes 12:19:23 my plan is that 12:19:30 still on that though 12:19:31 ok let me know if you need the data in a different format 12:19:44 I know t his is painful.. but we are figuring out a lot of things at the same time 12:19:47 is good as it is in extra_info_bandwidth now 12:20:00 yes, well, it happens :) 12:20:45 oook! 12:22:22 any other issue we should discuss? 12:22:33 nothing from my side, thanks 12:22:34 i'm fine 12:22:41 otherwise if everyone is good we can end the meeting 12:22:44 * hiro is groot 12:23:45 #endmeeting