11:46:49 <buxy> #startmeeting
11:46:49 <MeetBot> Meeting started Wed Aug 28 11:46:49 2013 UTC.  The chair is buxy. Information about MeetBot at http://wiki.debian.org/MeetBot.
11:46:49 <MeetBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
11:47:27 <buxy> #topic review of last week
11:48:12 <buxy> mlalic: so what happened this week? it looks like the schedule didn't work out
11:48:33 <mlalic> As you can probably tell, this week didn't really go all too well. My largest issue was that I spent *a lot* of time on the package dependency "side story"
11:48:58 <mlalic> On first sight, it doesn't look that complicated
11:49:43 <mlalic> But there are quite a bit of edge cases, some that I am not even sure how to handle, when using the new PTS's models
11:50:50 <buxy> Care to elaborate a bit?
11:52:05 <mlalic> For example, a binary package name can be associated to different source package names, depending on the repository it is found in. If there is a source package which has a dependency on that binary, it sort of seems wrong to show it as dependent on both the source packages
11:52:51 <mlalic> So the source-to-source dependency shouldn't match SourcePackageNames, rather SourcePackages?
11:53:06 <mlalic> (unlike what we discussed last time...)
11:54:41 <buxy> a dependency is on a SourcePackage but it points to a SourcePackageName
11:55:11 <mlalic> Further, when updating the repository information from Sources/Packages files, the dependency update needs to happen after the initial update since some packages can use newly added binaries as dependencies. It isn't such a big problem to create a new task for this, but then that one also ends up having to iterate through all Sources and Packages, since we decided not to keep the raw data in the database. I was trying to find a more efficient way of doi
11:55:12 <mlalic> ng it...
11:56:50 <buxy> Or the current task could be extended to that 2-pass work by itself.
11:57:18 <mlalic> Overall, I don't know, it really shouldn't be that hard to accomplish, but somehow something just wasn't clicking for me and then I started getting frustrated which didn't help :(
11:57:53 <buxy> You should have spoken to me, we could have left that story on the side for clarifications and continue with the rest.
11:58:55 <buxy> mlalic: can you push what you currently have for that story in separate branch for inspection? (or send me patches to review)
12:01:00 <mlalic> The only thing that really works there is updating BinaryPackage instances after the SourcePackage instances. I can push that, but it isn't that interesting since it doesn't have much to do with the dependency part
12:02:03 <buxy> so you were stuck at the design phase and there's no code at all (even if not working)?
12:02:49 <mlalic> I have the code which resolves the dependencies between source names which works the same as the current PTS
12:03:22 <mlalic> But I couldn't settle on a new model for these dependencies
12:04:08 <mlalic> In the current PTS, sources are said to depend on other sources without considering the repository they are found in
12:04:24 <buxy> push what you have (but not on develop), it's still a useful base to have a conversation about it
12:05:44 <mlalic> Okay...
12:06:40 <mlalic> Another thing there: in the beginning when we discussed the models, I was worried that the BinaryPackage and SourcePackage relationship seemed denormalized. Namely, it could happen that a BinaryPackage points to a SourcePackage which is not in the same repository
12:06:49 <mlalic> Which kind of doesn't make sense, does it?
12:07:15 <buxy> #action mlalic to push his work in progress on "WNPP derived information" so that we can have a conversation on how to best go forward and what the proper model is
12:07:39 <mlalic> Yeah, it doesn't happen if the code/"application layer" properly does the updating, but the database does not enforce the constraint
12:07:59 <mlalic> Do you see this as a problem?
12:08:51 <buxy> No. The database can't enforce everything and I don't see this as particular problem.
12:09:41 <mlalic> Right, okay... Well, one of our goals with the model was to make it normalized, wasn't it, so that is why I am asking
12:09:42 <buxy> Did that side story really eat you ~4 days of work?
12:09:53 <mlalic> No, not 4 days
12:10:03 <mlalic> 1.5
12:10:51 <buxy> mlalic: yes, but repositories are not always normalized either :)
12:11:05 <mlalic> I lost out on one whole day of work on Thursday, unfortunately
12:11:40 <mlalic> And the rest was some spent on some of the stories from the planned log (when I got sufficiently frustrated) and the other todo tasks we had agreed on last time
12:12:37 <mlalic> (I had an end of the month test here on Friday, so I studied a bit for it on Thursday.)
12:12:44 <buxy> ok
12:13:25 <buxy> mlalic: so you did some research on sso.debian.org, what were your conclusions? I don't see any estimation on the corresponding story.
12:14:04 <mlalic> Yeah, I did, but I wanted to discuss some of the findings first
12:14:23 <buxy> Ok, let's do it then.
12:14:57 <mlalic> So, what seemed like the most Django-friendly way was to simply add another AuthenticationBackend, along with the default one. Django goes one by one and tries to find one which can authenticate the user
12:15:49 <mlalic> However, only on Django 1.6 is it supported to have one of them completely halt the authentication (no other backends are considered)
12:16:21 <mlalic> So yeah, firstly we shouldn't consider that option, I suppose?
12:17:35 <mlalic> More important questions are: should one PTS account be allowed to be associated with multiple debian.org emails?
12:18:52 <buxy> I'm not opposed to use AuthenticationBackend even it means that we have some double-authentication possible until we switch to Django 1.6.
12:19:16 <mlalic> Also, when a user tries to register with a debian.org email, it seems he shouldn't even get the confirmation email, rather he should be asked to simply log in on sso and then create the account automatically on PTS. (last time we mentioned that he'd get the confirmation mail, but wouldn't be prompted for the password)
12:19:59 <buxy> mlalic: why the question for multiple @debian.org?
12:20:04 <mlalic> However, when the user first logs in with the debian credentials, I think he should be asked to merge the account with an already existing PTS account?
12:20:31 <buxy> I don't see a good reason to do that but it could happen in some rare cases (aliases when your acount got renamed)
12:21:05 <mlalic> Mostly due to the way the account "merging" would be implemented. If we want to forbid multiple @debian.org accounts, additional checking needs to be implemented there
12:23:22 <mlalic> And I think the last thing here is about testing. Unit testing seems like it should be fairly okay, since the authentication is based on a request.META REMOTE_USER property, but I'm wondering if we need any integration tests with apache here?
12:23:49 <buxy> I don't mind the specific workflow. If you feel like proposing this on first login, why not... but it should be doable later whenever one needs it (obviously he should be required to authenticate against the merged account before proceeding with the merge)
12:25:06 <mlalic> Yeah, of course authentication would be required. But what do you mean about later on: simply adding a @debian.org mail to the account or merging any two PTS accounts, even with no @debian.org mails?
12:25:53 <buxy> I mean that when the user wants to add an email that is owned by another account he should be offered to merge both.
12:26:20 <mlalic> Ah, right
12:26:55 <mlalic> Well, we should make that a separate story or update the existing "associate additional mails", since that is not allowed as it stands
12:28:43 <buxy> Updated that story, please update your estimation.
12:29:58 <mlalic> Okay. Now in this case, perhaps asking for the merge when first logging in with a debian address is unnecessary?
12:30:13 <buxy> Yeah, that's why I mentionned it.
12:32:09 <mlalic> Okay, great. Yeah, there was one more thing about that. You mentioned pulling some information from LDAP, but the only documentation I found on how to access it seems to indicate that clients can only connect to it from debian machines?
12:32:22 <buxy> Hum, asking the user to authenticate might be problematic.
12:32:27 <mlalic> In any case, that part of the story seems best to be moved to a separate story
12:33:02 <buxy> If you want to add a @debian.org email, you'd have to authenticate via the SSO and this would immediately changes your identity and create a new email.
12:33:13 <buxy> Right.
12:33:33 <mlalic> Heh, yeah, this goes back to the question of multiple debian.org addresses
12:34:04 <buxy> What part of the story are you referring to here? the merge of the LDAP thing?
12:34:12 <mlalic> LDAP
12:35:11 <buxy> Before answering this on LDAP, shall we then decide that merging accounts is also done with a challenge email? i.e. you'd have to click on a link to confirm the merge?
12:35:38 <buxy> Is this safe enough?
12:36:22 <mlalic> If we assume users don't just click any link that shows up in their inbox and then add another prompt on the page it seems okay to me
12:37:07 <buxy> If any account with @debian.org are forced to login via the SSO, it also limits the risks.
12:37:47 <mlalic> Yes, that is true too
12:38:08 <buxy> Right so this would require a double confirmation: clicking on the link, then authenticating as the user who requested the merge (if that was not yet the case)
12:39:47 <mlalic> Even if the user is already authenticated, I'd still keep a "are you sure?" prompt
12:40:49 <buxy> Yes.
12:41:46 <mlalic> That sounds safe enough, I think...
12:41:51 <buxy> Indeed.
12:43:48 <buxy> Ok. So can you put an estimate on the SSO login story now?
12:44:09 <mlalic> Are we keeping LDAP as part of it then?
12:45:25 <buxy> Oh LDAP. So the PTS will run on a debian.org machine when in production.
12:46:16 <buxy> So we certainly could make use of that feature. I wonder if we don't have an anonymous access to some for the public information that are already available on db.debian.org anyway.
12:46:32 <buxy> there's also finger@db.debian.org that could provide the required information
12:46:43 <buxy> finger <login>@db.debian.org
12:46:54 <buxy> but I wouldn't want to use that in the PTS :-)
12:48:06 <buxy> mlalic: we have enough public info available from non debian.org machines as well
12:48:18 <buxy> try ldapsearch -x -H ldap://db.debian.org -b dc=debian,dc=org uid=hertzog
12:49:12 <buxy> so yeah, I'd keep it, it should be trivial
12:50:12 <mlalic> Mhm, okay
12:50:26 <mlalic> I didn't mean ditching it completely either way, just whether it should be in the same story :)
12:51:03 <buxy> I think it wouldn't change more than 0.5 points of story so...
12:51:29 <mlalic> And yeah, something you probably missed in the moment: do we need some sort of apache integration tests or do we assume that the necessary REMOTE_USER is correctly set?
12:52:17 <buxy> mlalic: I don't see how we would do those apache integration tests... non-automatic tests are not very useful.
12:52:36 <buxy> so I guess we'll assume that this part works and is properly setup
12:53:41 <mlalic> Okay then, I think everything is covered
12:54:57 <buxy> Good.
12:55:11 <buxy> mlalic: is the email -> news thing working well now?
12:55:17 <mlalic> It seems so
12:55:24 <mlalic> I've been checking it and there are already generated mails
12:55:32 <mlalic> s/mails/news from mails
12:56:22 <buxy> mlalic: BTW, the news should be in revers order in the list
12:56:28 <buxy> latest on the top
12:57:16 <mlalic> Oops! Yeah, will fix it
12:57:49 <buxy> hum, http://pts.debian.net/news/6755 -> looks like some <pre>-like formatting is missing?
12:58:48 <buxy> it should really be in a mono-spaced fonts and the multiple spaces should be preserved
12:59:11 <buxy> #action mlalic to reverse the order of news (latest first)
12:59:27 <mlalic> Yeah, it's easily fixed too :)
13:00:10 <buxy> #action mlalic to improve the rendering of news generated by email (<pre></pre> for mono-spaced font and spaces preserved)
13:00:58 <mlalic> BTW, the configuration with multiple .forward files looks quite nice to me! No idea why it's not more common practice in exim...
13:01:58 <buxy> actually it's a reminiscence of the days where Debian was using qmail on servers
13:02:30 <buxy> mlalic: did you had some feedback on your blog posts?
13:02:45 <mlalic> Yep
13:03:25 <mlalic> Enrico Zini suggested to maybe have all mails delivered to a spool and then have a single consumer process which dispatches/processes them
13:03:55 <mlalic> Although he agrees not to go messing with it while the current system load average setting keeps us from crashing
13:04:12 <mlalic> I've also had some feedback regarding the new action items
13:04:41 <mlalic> Namely, that some of them seem less useful now, like lintian and log checks not directly providing the numbers in the short description
13:05:19 <mlalic> I'm waiting on Stefano and you to comment on the action items for the moment and then we can account for such feedback
13:05:55 <buxy> right, feel free to take some initiative when you get such feedback, we can always revert if it's not good
13:06:55 <buxy> mlalic: did you update your tries to record rss/size instead of vsz?
13:07:02 <mlalic> Ah yes, I have
13:07:24 <buxy> What does it give?
13:07:29 <mlalic> between rss and size there isn't much difference when the same database is being used (as in only a few KB)
13:08:02 <mlalic> And the difference between the sqlite and posgres also disappears into a very slight margin of only 1 MB
13:08:17 <buxy> Ok, that makes sense then.
13:08:18 <mlalic> The new PTS uses about 30 MB whereas the old one is at 9
13:09:14 <mlalic> Still, from what I understand, the extra libraries could decide to actually use all the memory reported by vsz? But we should be safe since the benchmarks show that they don't?
13:09:24 <buxy> It's just that postgreql related libraries are bigger than the sqlite libraries and need more virtual address space. But in practice the difference is negligible.
13:10:18 <mlalic> I was thinking of adding in an edit with these new results to the blog?
13:10:32 <buxy> Yes, please do. And enable comments. Can you do that?
13:11:09 <mlalic> Ah yeah, they are enabled in the settings for all users
13:11:19 <mlalic> I don't know what causes the problems...
13:11:58 <mlalic> Everything looks the same as in FAQs I could find on the matter
13:12:44 <buxy> Maybe it will only apply for new posts? and you have something special to do for old ones? (or maybe it's not possible for old ones)
13:13:15 <buxy> Not a big deal but you're surely missing a lot of feedback due to this which is a bit of a pity.
13:13:22 <mlalic> I don't think so since there are post-specific settings as well, which also show that anonymous users can comment
13:17:52 <buxy> #action mlalic to update his former blog post with updated information about memory usage
13:18:41 <buxy> I think you never sent that mail to any exim list asking for another way to rate limit delivery on a specific transport (or email).
13:19:57 <buxy> It would be still worth doing it IMO because it feels a bit clumsy to rely on the load average setting to avoid problems. In particular when you set the setting much lower than what Debian does.
13:20:35 <mlalic> Oh, I thought that we should just keep this until it'd prove failing. Anyway, yeah, I'll do it then
13:21:14 <mlalic> you are referring to pkg-exim4-maintainers@lists.alioth.debian.org I suppose?
13:22:31 <buxy> Well set it at "queue_only_load = 35" like Debian does.
13:22:53 <buxy> mlalic: no, an upstream support list exim-users or something like that
13:23:16 <buxy> And we'll see if it's enough to avoid problems.
13:24:23 <buxy> Debian also has this:
13:24:26 <buxy> message_body_visible = 5000
13:24:27 <buxy> queue_run_max = 50
13:24:28 <buxy> deliver_queue_load_max = 50
13:24:28 <buxy> queue_only_load = 35
13:24:29 <buxy> smtp_load_reserve = 20
13:24:55 <buxy> #action malic to update the exim config to get closer to the Debian in terms of load limit for Exim
13:25:35 <buxy> http://www.exim.org/mailman/listinfo/exim-users
13:26:01 <mlalic> Yeah, I found it too, thanks
13:28:12 <buxy> #action mlalic to seek advice on exim-users mailing list on how to try to rate-limite deliveries for a specific router/email/domain
13:28:23 <buxy> I think we're done reviewing the past.
13:28:29 <buxy> #topic next week planning
13:29:09 <buxy> Since the last iteration wasn't completed I only added stories...
13:29:18 <buxy> two stories
13:30:05 <buxy> I moved the WNPP derived info from "in progress" back to "planned" so that we have a bit of time to discuss by email the details on how to achieve it (once you have sent me the current code for review)
13:30:20 <buxy> pabs: do you expect to start contributing to the PTS this week?
13:31:21 <mlalic> buxy: Right
13:32:12 <buxy> mlalic: I have informed obergix (Olivier Berger) that we're not going to tackle RDF support in the rewrite and I invited him to contribute this feature if he could, he has no prior Django experience so he might ask you (and me) some questions. I tried to give him some initial directions. Try to answer his questions if he gets in touch.
13:34:05 <mlalic> Yes, of course. He's already contacted me about where to find the documentation (he wasn't on the develop branch) which I've cleared up. Of course, any other questions and feedback is welcome :)
13:34:10 <buxy> mlalic: does the planning look workable to you?
13:34:52 <buxy> mlalic: ok, please put me in CC in those discussions
13:35:19 <mlalic> To be honest, I am hoping that the WNPP story does not take 8 points of effective work once we agree on the design matters via email. The original story estimate had more to it than just the dependency resolution...
13:35:43 <mlalic> Would you like to be included in all my answers to feedback or just in this case?
13:36:17 <buxy> mlalic: BTW, you should merge to master what's deployed, or we should just ditch develop and use master because there's no useful distinction currently
13:37:15 <mlalic> Fine, we can lose develop then... I suppose it's more confusing than useful for the moment
13:37:36 <buxy> mlalic: in all feedback would be better indeed
13:38:15 <mlalic> No problem
13:38:25 <buxy> Yeah, we can always reintroduce it when we run in production on debian.org but I doubt we will need it.
13:39:34 <buxy> Yeah I also hope that WNPP takes less than 8, but you have 3 more user stories waiting at the top of the backlog if you manage to make up a little of the... backlog :)
13:40:37 <mlalic> Sure... As for the rest of the planned stories, it all looks good. I have no issues there
13:40:46 <buxy> mlalic: BTW I guess that Stefano would also like to be in CC like me
13:41:06 <buxy> #topic misc questions
13:41:21 <buxy> mlalic: does that mean that you don't have any remaining question?
13:41:40 <mlalic> I do have something, but it's not related to these stories
13:41:51 <buxy> go ahead then, just ask
13:42:31 <mlalic> I've been wondering whether we should perhaps fix the directory structure regarding templates and static assets. Right now, the setup uses the template loader which looks under the apps' templates directory
13:42:55 <paultag> (I've got a question)
13:42:58 <mlalic> But perhaps it'd make more sense to have a top-level templates/static where all assets are kept? Not sure about it
13:43:50 <buxy> What for?
13:44:23 <buxy> I added a cached template loader to improve perfs a little bit.
13:44:34 <mlalic> Mainly I thought that things like debian/templates/debian/ doesn't look too nice
13:44:42 <buxy> (hidden in the reorganization of settings)
13:44:50 <paultag> mlalic: (Thank you for all your work, by the way) - Is there any chance the changelog / copyright links can be re-added to the new PTS? I've been using pts.d.n primarally now, and miss them  (feel free to finish current thread, but I need to run) - thanks again, I love the new PTS
13:45:18 <buxy> paultag: they are there, look in versioned links on the left
13:45:34 <themill> paultag: bottom left
13:45:38 <paultag> Oh, I see
13:45:41 <mlalic> paultag: Thanks for your encouragement, as buxy says, the files are there :)
13:45:43 <paultag> I use C-f to find links
13:45:44 <themill> Oh btw your question about whether the excess whitespace dilutes the information too much -- I'd say it does a bit.
13:45:48 <paultag> so I missed groking the icons
13:45:50 <paultag> but great, thanks y'all
13:46:56 <mlalic> themill: Would you say it is primarily to the space between the panels or the padding in the panels themselves (or actually maybe both? :))
13:48:03 <themill> bit of both; most of the space is taken up by whitespace around the text not space between panels though (just becuase there are lots of lines of text and not many panels).
13:48:06 <mlalic> buxy: Oh and another reason for the templates. Things like 404.html or 500.html don't really seem to belong to any app
13:48:20 <buxy> mlalic: we could centralize them I guess, except for apss that we expect to be useful outside of the PTS (like the authentication one)
13:48:32 <themill> The text is certainly more readable with extra whitespace around it; I just miss being able to see the entire page in one screenful.
13:49:19 <buxy> We should definitely reduce the header as well to compensate a bit.
13:50:53 <buxy> mlalic: and for static assets it's the same story IMO
13:51:10 <themill> I assume the "was removed from unstable" comment in http://pts.debian.net/cython is a problem in the news feed
13:52:47 <mlalic> buxy: This is why I wasn't quite sure about it -- there seem to be some things better suited to be in the app and others in a common top-level directory. The 404.html template being an example of the latter and the panel templates of the former
13:53:35 <mlalic> themill: I am not sure it is a problem. If this package version used to be found in unstable in the same time as the current one, but was subsequently removed from the repository, this is the expected news item
13:54:20 <mlalic> themill: Do not it is not the same as the entire package being completely removed from unstable, only the one version. Does it make sense?
13:54:36 <mlalic> s/Do not/Do note
13:55:05 <buxy> mlalic: it's unexpected at least, it's not really useful for us
13:55:06 <themill> it's just one arch that is removed? Perhaps the news item could say that? (and the link could say something other than "None")
13:56:32 <themill> Are the news items also in ASC not DESC order there?
13:56:53 <mlalic> buxy: So we should only have a news item when the entire package is removed from the repository?
13:57:01 <buxy> mlalic: https://trello.com/c/luvx3NK8/63-generate-news-from-changes-in-the-repositories-monitored-8 didn't specify anything for removals of one version, only for complete removal
13:57:18 <buxy> "the name can no longer be found in the repository"
13:57:39 <themill> Can the bls link be to http://qa.debian.org/bls/packages/c/cython.html not http://qa.debian.org/bls/index.html ?
13:57:45 * themill is getting really picky now
13:58:11 <themill> It's great that it's looking so good and working so well that we can worry about little things like this.
13:59:22 <buxy> themill: even better would be if you tried to send the corresponding patch :-)
13:59:32 <themill> hehe
13:59:35 <buxy> the goal of the project was to make it easier for people to contribute ;)
13:59:46 <mlalic> themill: I think that link is already there
14:00:32 <buxy> mlalic: the link (in the long description in the popup) points to http://qa.debian.org/bls/index.html  when he would like to have it point to http://qa.debian.org/bls/packages/c/cython.html
14:00:39 <themill> buxy: yeah, looking at this more is on my "one day in the future" list.
14:00:56 <mlalic> Ah, in the action needed item
14:01:07 <buxy> and even better would be to have the link in the short description too
14:01:23 <themill> ah yes, sorry, I should have been more specific about it being the action popup. I didn't realise it was also in the links box.
14:03:31 <buxy> mlalic: another detail, the links in binaries don't point to packages.debian.org like they should
14:04:15 <buxy> it's great to see external feedback arriving
14:04:26 <buxy> anyway time is over for the meeting
14:04:33 <buxy> mlalic: anything else on your side?
14:05:08 <mlalic> Yeah, I think this was pretty much it
14:05:21 <mlalic> Would you mind adding the last two things to the minutes
14:05:34 <mlalic> Oh, actually, is #action for everyone or just the chair?
14:05:39 <buxy> mlalic: do it yourself, everybody can do it
14:05:47 <mlalic> Yeah, just realized, sorry
14:06:22 <mlalic> #action mlalic to add links in the binaries panel to packages.debian.org
14:06:50 <mlalic> #action mlalic to improve short description of logcheck action item
14:08:18 <buxy> #action mlalic to fix the automatic news to not generate news when each source package version is removed from a repository but only when all version of it are gone
14:08:58 <buxy> #stopmeeting
14:09:03 <buxy> #endmeeting