Working Group Minutes/EWG 2011-11-14

From OpenStreetMap Foundation

Attendees

IRC nick Real name
apmon Kai Krueger
RichardF Richard Fairhurst
TomH Tom Hughes
zere Matt Amos

Summary

  • apmon has created a PPA for rails_port development at [1] which pulls in the rails3 source.
    • Some discussion of the performance regression seen with rails3.
    • RichardF and zere to test packages by next meeting.
  • Board meeting stuff was pushed back, as the minutes haven't been published yet.
  • Discussion of the upcoming hack weekend in London [2] and having an online event over the same period to include those who can't be present in-person.
    • RichardF offered CWG's help in promoting the event, which was gratefully accepted.

IRC Log

18:01 < apmon> Hello everyone
18:01 < zere> last meeting minutes http://www.osmfoundation.org/wiki/Working_Group_Minutes/EWG_2011-11-07 if there's anything objectionable, let me know.
18:02 < TomH> yo
18:04 < apmon> Do we have an agenda?
18:04 < RichardF> hello
18:09 < zere> yep. let's go through the last minutes' actions - RichardF everything OK with the CWG stuff?
18:10 < RichardF> yep. all good.
18:12 < apmon> I have created the ubuntu rails-port package
18:12 < apmon> although it doesn't really deserve the name package
18:13 < zere> cool! and that's up on launchpad?
18:13 < RichardF> excellent!
18:13 < apmon> It basically consists of two files: The control file, specifying the ubuntu package dependencies, and the postinstall script that does everything else
18:13 < apmon> https://launchpad.net/~kakrueger/+archive/openstreetmap
18:13 < apmon> The openstreetmap-rails-port package
18:13 < zere> awesome. does it pull the sources down from git.openstreetmap.org?
18:14 < apmon> currently pulls sources from TomH's rails3 github branch
18:14 < apmon> Once the rails3 is merged back into the main branch on git.osm.org I'll switch it to that
18:15 < zere> i thought that was merged into the master now?
18:16 < zere> nope. ok. i guess not.
18:16 < apmon> I thought it was backed out again, as there were some failures
18:17 < apmon> The number of commands needed to get things up and running is actually really quite short
18:17 < apmon> the rails bundler seems to take care of most things nicely
18:18 < TomH> git clone; gem install bundler; bundle install
18:18 < apmon> pretty much
18:18 < apmon> On ubuntu, you need to update the rubygems first, but apart from that
18:20 < apmon> Which is why calling what I did a "package" a bit overstated
18:22 < apmon> TomH: Did you manage to get a handle on the increased CPU load on the front end servers due to rails3?
18:24 < TomH> not really, because I have no real way to look for that
18:24 < apmon> I have read, that rails 3 might cause a significantly larger overhead on the ruby garbage collector which results in a overall slow down
18:24 < RichardF> apmon: I guess all we need to do now is document it on the wiki? Let me know if you need a hand :)
18:24 < TomH> I have sorted a faster js though
18:24 < RichardF> then we can announce on various blogs etc.
18:24 < TomH> apmon: early rails 3 was said to be slower, but I thought more recent versions had fixed that
18:25 < apmon> RichardF: Do you have a Ubuntu (Oneiric) VM, you could test it out on
18:25 < apmon> I am not sure,  I think some of the performance issues were fixed, but not all
18:26 < RichardF> apmon: I don't, I'm afraid. I can see if I can work out how to set one up though it won't be until the weekend when I have a working Internet connection and a suitable machine :)
18:26 < zere> any mileage in moving to ruby 1.9.2 - isn't that supposed to have a better GC?
18:26 < apmon> The info on the garbage collector overhead comes from the RailsConf 2011 presentation by Aaron Patterson
18:26 < apmon> http://www.youtube.com/watch?v=kWOAHIpmLAI (towards minute 35)
18:27 < TomH> zere: well it's not available on lucid
18:27 < TomH> rubyee is supposed to be better but I accidentally tested that and it seemed slower
18:30 < apmon> It might well be that the GC overhead is an unrelated issue
18:32 < zere> is the CPU use increased to the point of needing another front-end node?
18:32 < apmon> I think the CPUs were maxed out on the two front ends, so probably yes
18:33 < TomH> not quite maxed out
18:34 < apmon> RichardF: It would be good if some more people could try it out before announcing it.
18:34 < RichardF> apmon: ok. should we ask for volunteers on dev@, or #osm, maybe?
18:35 < zere> apmon: ok. i'm happy to test it out. looks like RichardF might be too at the weekend. how many volunteers is a good number?
18:35 < apmon> If the two of you are happy, that is probably sufficient
18:35 < apmon> It is not like there is a huge mass market for the rails_port anyway... ;-)
18:36 < zere> hopefully there is a lake of untapped talent behind the dam of difficult installation ;-)
18:37 < apmon> The hugely difficult installation of "git clone; gem install bundler; bundle install"
18:37 < apmon> So I am wondering if having to add another repository might not end up being more difficult than the installation itself... ;-)
18:38 < RichardF> not that the package helps, but if you think that's easy, you clearly have never tried to run bundler on a PPC Mac ;)
18:38 < zere> so i was hoping to relate some stuff from the board meeting, but we still haven't published the minutes. not sure whether the right thing to do is to talk about it yet, or wait for the official announcement.
18:38 < zere> i'm going to play it safe and wait for the minutes to be released. there's a few things of EWG interest in there.
18:39 < zere> apmon: do you use the system (ubuntu packaged) ruby-libxml and rmagick?
18:39 < apmon> well, mkl's post has cause quite a stir already, so it might be ok to add to it
18:40 < apmon> zere: Yes
18:40 < TomH> zere: that won't generally work because they aren't gems, so rails won't find them
18:41 < apmon> The list of dependencies I have is the following: ruby, rdoc, ri, irb, libxml2-dev, libxslt1-dev, ruby-dev, libmagickwand-dev, build-essential, libruby, git, postgresql, postgresql-contrib, rubygems, libmemcached-dev, libsasl2-dev, libpq-dev, rails, nodejs
18:41 < apmon> With TomH's most recent commit of including therubyracer, nodejs might no longer be necessary
18:41 < zere> ok. i was just wondering about alexandrescu on the dev ML. he somehow managed to get a segfault out of his rails_port.
18:42 < TomH> apmon: nodejs is not ncecessary and doesn't even exist on ucid
18:42 < zere> alexandru, even.
18:42 < apmon> I haven't done super extensive testing of the packages, so it might segfault under some conditions too
18:43 < apmon> TomH: Some part (I think it was rake db:migrate) needed a js-engine. nodejs was one that worked.
18:44 < TomH> apmon: AFAIK only building assets needs it but I might be wrong
18:44 < apmon> other js-engines are likely to work just as well. But a clean oneiric installation doesn't have one by default, so I needed to include some dependency
18:44 < TomH> we were usng rhino as it's the only js command line on lucid, ,but it's horribly slow
18:44 < TomH> hence using therubyraceer instead
18:45 < TomH> takes like one minute instead of six to build the assets
18:45 < apmon> That is quite a difference!
18:46 < apmon> zere: Is the main thing concerning the ewg from the board meeting the frontpage design?
18:47 < TomH> well rhino is written in java so it fires up the jvm every time it wants to eval soem js
18:47 < TomH> trr is a ruby library that calls direct into v8
18:47 < zere> apmon: yes, that's the main item. there's another couple of items too, but that's the biggest one.
18:50 < apmon> OK, will be interesting to read what you guys have come up with, with respect to that
18:53 < apmon> Anything else for this meeting?
18:53 < zere> there's really nothing we've "come up with", it's more about having EWG take a role in ingesting, reviewing and fixing patches coming from the design process.
18:58 < zere> ok... is there anything else anyone wants to discuss?
18:58 < RichardF> do we have anything we need to do (or ask CWG to do) re: hack day?
18:58 < zere> if not: agenda for the next meeting is discussing the stuff from the board f2f (assuming we've released the minutes...)
18:59 < RichardF> e.g. getting publicity, encouraging people to come, talking about what we might be working on...?
18:59 < apmon> Is there anything that ewg can / should do for the hack-weekend?
19:00 < apmon> RichardF:  Arg, I see, I overlooked your comment
19:00  * RichardF laughs
19:00 < RichardF> do we want to encourage remote hacking? i.e. "even if you can't get to London, spend the day hacking, and there'll be lots of friendly faces in #osm-dev to help with any problems you might have"?
19:01 < apmon> RichardF: Yes, as that is the way I intend to join... ;-) And the more remote hackers the better
19:02 < zere> we talked about an e-hack day before. iirc, the consensus was that it wasn't best to run it contemporaneously with the RL hack day?
19:04 < zere> yeah, http://www.osmfoundation.org/wiki/Working_Group_Minutes/EWG_2011-08-29
19:04 < zere> "possibly pure e-hack. Otherwise people in a RL meeting aren't inclined to communicate via irc"
19:04 < zere> although i'm certainly willing to give it a try, if you are, apmon?
19:08 < apmon> Assuming the timezone difference isn't going to cause too much sleepiness, I will try and be on irc during the time trying to hack on things
19:09 < zere> 24h rolling hackathon!
19:10 < zere> ok. so, yes, RichardF, we would welcome CWG mentioning the hack weekend in its real-life and IRC incarnations.
19:12 < zere> thanks to everyone for coming. see you next week!