Working Group Minutes/EWG 2012-10-22

Attendees

IRC nick	Real name
apmon / apmon_	Kai Krueger
jfire	John Firebaugh
ppawel	Paweł Paprota
shaunmcdonald	Shaun McDonald
tmcw_	Tom MacWright
zere	Matt Amos

Summary

• Arising from previous meetings:
  • ppawel reported on Clickable POI research [1]
    • The overpass demo clickable site is at: [2]
    • And the openlinkmap equivalent is at: [3]
    • There was a long discussion on various methods of implementing clickable things, but without consensus - leave this item until next week
  • ppawel reported no change on help.osm.org item since last week.
• Routing backend
  • AGREED: we now have access to OSRM and gosmore servers, so it seems this task is over until we get more experience from running the frontend.
• Notes branch
  • Issues of concern: whether to allow anonymous logins; whether to attempt to import the existing external OSB database.
  • Login: If anonymous users are allowed to enter bugs, then a potentially complex system of anti-spam measures may be necessary.
  • Transitioning the OSB database - it's not clear if (depending on the login issue) the databases may need to run in parallel for a while, making it even harder to merge in the future.
• AoB
  • Flipped a coin for time-change: the meeting will stay at the same time US-local, meaning next meeting is at 18:00 GMT/UTC.

IRC Log

18:03:40 <zere> minutes of the last meeting: http://www.osmfoundation.org/wiki/Working_Group_Minutes/EWG_2012-10-15
18:04:03 <zere> let's go over the actions from last meeting
18:04:19 <zere> first, ppawel to look into the various alternatives on the clickable POIs task
18:04:28 <zere> anything to report here?
18:04:43 <ppawel> I sent a message to the dev list today - not much time spent on it, wiki page in progress
18:05:18 <ppawel> http://wiki.openstreetmap.org/wiki/Top_Ten_Tasks/Progress/Clickable_POIs_on_the_frontpage\
18:06:14 <ppawel> I think the main decision with this task is to choose what database/service/API is on the server side
18:06:31 <ppawel> the frontend part is kind of separate from backend
18:07:15 <zere> yeah, it all flows through from the intersection of what's actually wanted vs. what's practical
18:07:38 <ppawel> plus frontend right now is tricky - I think jfire is working on leaflet port
18:08:00 <ppawel> so if any implementation for POIs should start then it probably should be based on leaflet already
18:08:30 <ppawel> as for backend - overpass api seems to me like a big dependency
18:08:44 <ppawel> a lightweight database containing only POIs would suffice
18:08:59 <zere> i'm not sure the choice of leaflet vs. OL makes much of an impact on the front-end... i might be wrong, but i think they're probably similar enough that porting wouldn't be a massive issue.
18:09:16 <ppawel> yes I think they have very similar feature set
18:10:10 <zere> there's another thing, though - do we just want POIs? the overpass version is capable of looking up all kinds of nearby features - roads, buildings, etc...
18:10:11 <jfire> porting to leaflet is largely linear effort based on the size of code… haven't found anything that was particularly tricky
18:10:29 <jfire> feature sets are pretty much the same, as ppawel says
18:10:53 <ppawel> zere, yeah that's another question..
18:11:16 <zere> yeah, so it seems that the big issue is really figuring out what this task actually needs / what's actually useful. from that we can start to make technology decisions
18:11:16 <ppawel> not sure if looking up roads make sense? maybe it does... see surface, restrictions... anything that's not rendered
18:11:28 <apmon> Imho, it would probably be best driven from the osm2pgsql db that is already doing the rendering
18:12:14 <zere> if the purpose is to make the data more interactive, then we're looking for enough to fill a small info-box with a forwarding link to a /browse/ page
18:12:21 <ppawel> postgis database can be easily hooked up to a WFS server and WFS can serve GeoJSON
18:12:30 <ppawel> leaflet has a plugin for GeoJSON and hovering over stuff
18:12:42 <ppawel> so technically it's simple - but is it scalable? don't know
18:12:55 <zere> from my point of view, a WFS server seems to me like a big dependency ;-)
18:13:21 * tmcw_ OH GOD NO NOT WFS
18:13:22 <ppawel> well it's a question of what do you mean by "big"... it's a standard way of serving geometry
18:13:30 * tmcw_ dies
18:13:43 <ppawel> but in OSM not a lot of things are standard so maybe it's not a good option
18:13:57 <tmcw_> We're talking clickable pois?
18:13:58 <zere> if we just want geojson then we should just serve "select as_geojson(...) ..." from the rendering database, no?
18:14:17 <ppawel> well that's certainly one way to do it..
18:14:33 <zere> tmcw_: clickable <mumble> at the moment ;-)
18:14:52 <tmcw_> So... I'd recommend first looking into mapnik/utfgrid
18:14:56 <zere> wondering whether it's worth restricting it to POIs.
18:15:07 <ppawel> anyway I think more important than technology is really defining what should be implemented
18:15:20 <ppawel> so what zere said earlier..
18:15:23 <tmcw_> It's likely a lot simpler/faster than doing this at the postgis level where you'd be treated to the same filtering & placement problems that mapnik is doing design-side
18:15:58 <zere> ah, we went over this a bit last time - mapnik/utfgrid seems like a good solution, but actually no-one has done the work to make it possible. so we're left looking at things that we can actually do
18:16:17 <tmcw_> Hm?
18:16:29 <tmcw_> Can you be more specific about 'no-one has done the work to make it possible'?
18:16:30 <ppawel> I've had good experience with WFS on contract work.. mapnik/utfgrid - no idea really
18:16:49 <zere> the idea with overpass is that you would't do the filtering and placement - allowing users to get accesss to information about features which have been occluded or aren't in the rendered style.
18:17:28 * tmcw_ not clear on why the mapnik/utfgrid option isn't the clearest here
18:17:48 <tmcw_> the tile server is already running mapnik 2, and there's a leaflet connector
18:18:01 <tmcw_> there's the question of how/which attribute data to bring over, sure
18:18:06 <zere> 'no-one has done ...' - there's an overpass demo site, so implementing this in osm.org is some straightforward javascript. mapnik/utfgrid is out there somewhere, but needs stuff putting into the style to make it work, etc...
18:18:06 <jfire> you really want the clickable stuff to align with the base tiles though, and if we end up doing something like the rail station merging described here http://mapbox.com/blog/processing-osm/ that is an issue with non mapnik/utfgrid solution, correct?
18:18:16 <apmon> I wanted to look at this years GSoC project for json vector tiles ( http://wiki.openstreetmap.org/wiki/Google_Summer_of_Code/2012/Data_Tile_Service ) but didn't get around to it.
18:18:51 <tmcw_> we should really separate "making all data visible to clients" and "making pois/other stuff clickable"
18:19:21 <tmcw_> the former brings in the classic "zoom to z6 and see everything / browser downloads 3gb fail"
18:19:48 <ppawel> I think no one is suggesting that - the question is what should be clickable
18:20:05 <ppawel> clickability doesn't necessarily mean downloading tons of stuff
18:20:40 <ppawel> http://overpass-api.de/open_layers_popup.html
18:20:41 <ppawel> like here
18:20:51 <ppawel> it's just a question what do you have in the database
18:21:19 <zere> yes, i'm not sure what the purpose of this is - the task description mentions POIs specifically, but also discovery of "rich metadata" and "depth of coverage beyond what is possible with two-dimensional cartography"
18:21:20 <apmon> Has anyone looked at how http://www.openlinkmap.org/ works?
18:21:36 <ppawel> apmon, yes I have described it briefly: http://wiki.openstreetmap.org/wiki/Top_Ten_Tasks/Progress/Clickable_POIs_on_the_frontpage
18:21:39 <zere> apmon: it's mentioned on ppawel's progress page
18:21:57 <ppawel> http://www.openlinkmap.org/api/list.php?bbox=19.317753314972,51.74392446232,19.648416042328,51.825427518256
18:22:40 <apmon> If clickable POIs is all that is wanted, then openlinkmap seems to fullfill that requirement quite nicely
18:24:03 <ppawel> on GUI side - yes, but my concern would be about backend side
18:24:23 <ppawel> overpass option looks nice but is it acceptable to have overpass server as a dependency?
18:24:39 <apmon> Imho, anything that isn't cached is a bad idea for the main page.
18:24:55 <apmon> So overpass would be a bad idea if this is to scale
18:25:29 <zere> oh, i think that's ok - it's a pretty stable bit of software. whether we run it in-house or rely on the external server is not an issue at this point.
18:26:08 <zere> sure - ideally we'd have vector tiles... (and without crossing into the next action) that solution doesn't really exist.
18:27:50 <ppawel> I guess the only next action for this TTT is to do more digging/thinking...?
18:28:32 <zere> big question: is the action of clicking either 1) "what is *this* thing?" or 2) "show me things here"
18:29:28 <zere> if it's the former, then it's entirely tied to the rendering and we'll just park this task until someone does the mapnik work to include utfgrid stuff, and jfire's leaflet port is merged.
18:29:41 <ppawel> I think the most common use case is just clicking on a POI icon - users expect it to show a bubble with info
18:30:18 <zere> if the latter - "depth of coverage beyond what is possible with two-dimensional cartography" - then a cartographic approach is pointless anyway
18:31:05 <tmcw_> I don't think that "depth of coverage beyond what is possible with two-dimensional cartography" belongs anywhere near a "next actions list"
18:31:17 <ppawel> I wouldn't say that (1) is tied to rendering - as evidenced by openlinkmap and overpass for example
18:31:45 <tmcw_> Not clear on that one
18:31:53 <tmcw_> Like, openlinkmap right now gives you lots of bubbles
18:32:00 <tmcw_> which is fine, but what are these bubbles?
18:32:16 <zere> if it's not linked to the rendering, then you get the situation where what's rendered in the background layer doesn't match what's clickable
18:32:20 <tmcw_> so, you have two options: either you add icons to the bubbles and run a shadow-mapnik-style in openlayers or leaflet or whatever to maket hese links self-obvious
18:32:30 <tmcw_> or you try to make them match the rendering, make the hit areas transparent
18:32:35 <ppawel> zere, when would that happen?
18:32:59 <zere> either stuff in the background is obscured by the layer on top, or the list of hit-areas available doesn't match the list of rendered features
18:33:50 <ppawel> but both renderer and the clickable poi layer are from the same database - how could they be out of sync?
18:33:57 <ppawel> (problems with replication delay aside...)
18:34:27 <zere> because the renderer may not render some features as their labels would obscure or be obscured by other features
18:35:04 <zere> many major shopping streets in OSM show only 1-of-2 or 1-of-3 of the available features
18:35:56 <zere> if you remember the old "osmarender" map layer, which didn't have any sort of label collision filter, looked pretty awful in these busy areas
18:36:11 <ppawel> I wonder how google maps solves this...
18:36:13 <zere> i think in google-land it's called "red dot fever"
18:37:15 <apmon> We are at 35 minutes in now, are there specific things to do on this topic for next week?
18:37:16 <zere> if you inspect google maps, there's a bit of json coming down the wire which is either a hit-box or utfgrid-style hit-map
18:37:42 <zere> so, as far as i can tell, they're integrating clickable stuff with the rendering process.
18:37:57 <zere> apmon: good point.
18:38:28 <tmcw_> it's a hit-box
18:38:28 <zere> what can we do to figure this out? anything that can be done offline before next meeting?
18:38:48 <tmcw_> utfgrid took a slightly different approach (hit-maps) to support polygon shapes
18:38:58 <tmcw_> which is more similar to what fusion tables-driven maps do
18:39:21 <ppawel> zere, I can expand the wiki page with more options (utfgrid, check cacheability of solutions).. if this is useful
18:40:51 <zere> i think we'll keep coming back to this central question of server-side (overpass) vs. client-side (utfgrid / metawriter) click handling, won't we?
18:42:03 <ppawel> probably
18:42:08 <zere> is it important functionality for this feature to be able to discover data that's not visible in the rendered map?
18:42:20 <zere> maybe we need to do a poll on this or something?
18:42:27 <zere> reach a wider audience?
18:42:56 <ppawel> imho google maps/bing should be a model here - just simple POI clicking for the first milestone..
18:43:05 <ppawel> when you do a poll you will get dozens of ideas
18:43:20 <ppawel> which is nice, but well.. who's going to implement all this
18:43:34 <zere> i was thinking just that one question: "discover non-visible data? Y/N/dont-care"
18:43:40 <zere> well, quite
18:43:56 <zere> implementation difficulty is another aspect of the problem to throw in
18:44:10 <apmon> zere: What is next on the agenda? Notes branch?
18:44:38 <zere> we have 2 more actions to get through first ;-)
18:45:04 <ppawel> quick one - help.osm.org - no change since last week
18:45:17 <ppawel> (I was supposed to try the export/import approach)
18:45:26 <zere> ok, i'm going to mark this as "non-resolved" this week and we'll come back to it.
18:45:41 <ppawel> ok
18:45:48 <zere> #info long discussion on clickable things without consensus - leaving until next week
18:46:10 <zere> but, of course, if anyone wants to do more to pin down the various options it would be very much appreciated.
18:46:25 <zere> #info ppawel> quick one - help.osm.org - no change since last week
18:47:11 <zere> apmon: anything to report on the vector tiles stuff other than what you already said?
18:47:26 <apmon> No, unfortunately I didn't get around to it
18:47:57 <zere> no worries :-)
18:48:01 <zere> ok
18:48:02 <zere> final
18:48:17 <zere> final item on the TTTs review is routing backend, right?
18:48:49 <zere> #topic routing backend
18:49:27 <jfire> I merged master with apmon's branch here: https://github.com/apmon/openstreetmap-website/pull/1
18:49:28 <zere> at SOTM dennis luxen said his server is at 0.0001% capacity (or something very low like that) and basically offered it for the hammering.
18:49:35 <apmon> Well, there is an instance of gosmore running on osmf hardware now, and there is the world-wide daily updated OSRM server
18:50:26 <zere> so, rather than choose a routing engine, we have access to several? sounds good to me :-)
18:50:26 <apmon> Sounds like routing backend is "solved" for the moment then.
18:50:47 <apmon> At least until we have a frontend in production and more experience what is missing.
18:51:21 <zere> #agreed we now have access to OSRM and gosmore servers, so it seems this task is over until we get more experience from running the frontend
18:51:24 <zere> ok
18:51:40 <zere> #topic notes branch
18:51:49 <zere> apmon: you wanted to discuss the notes branch?
18:52:15 <Guest2780> The question is what are the next steps to get it ready
18:53:25 <Guest2780> Two things come to mind. 1) The question of login as already discussed on the mailinglists 2) The question of transitioning the old OSB database
18:54:26 <zere> the login proposition i saw seemed eminently reasonable: no login necessary to report bugs, but login necessary to close / comment / assign / etc...
18:54:43 <zere> what's the general feeling about that that you've picked up?
18:55:09 <apmon_> People do seem to prefer not requiring a login
18:55:25 <apmon_> The main issue I see with that is spam though.
18:55:39 <apmon_> Allowing anyone unckeced to post notes onto osm.org seems unwise
18:57:15 <apmon_> The consensus (at least on talk-de) was that the main point of OSB was to reduce the barrier to contribute and asking people to sign-up is too much of a barrier.
18:57:39 <zere> yup. i can see that
18:58:03 <zere> so how about traditional anti-spam measures? captchas and that sort of thing?
18:58:45 <apmon_> presumably only for non logged-in users?
18:59:22 <apmon_> But then someone would have to code that up.
18:59:58 <apmon_> I kind of like the suggestion of gravitystorm, to do a soft or beta launch, requiring people to log-in at first
19:00:28 <apmon_> and then find a solution for non logged-in users, once we know the notes system doesn't get overwhelmed as it is
19:00:42 <zere> that sounds sensible too.
19:00:55 <apmon_> Problem is, that interacts with any potential strategy of migrating the existing OSB
19:01:05 <zere> as for database migration.... i assume there's valuable stuff in the old OSB database?
19:01:20 <zere> oh, it had its own user system?
19:01:31 <apmon_> No
19:01:48 <apmon_> Well, only string base user names on the bugs, but that isn't too problematic
19:02:10 <apmon_> The issue is a potential for running both systems simultanious for a while
19:02:25 <apmon_> and then having to merge rather than migrate the DBs
19:02:43 <zere> is there lots of spam in the existing OSB database?
19:02:57 <apmon_> I don't know for sure, but I think it isn't too bad
19:03:10 <shaunmcdonald> Possibly even only soft launch with logged in users only, so the non-logged in users don't see the bugs at least initially
19:03:15 <apmon_> But osm.org would be much more of a lucrative target for spamers would be my guess
19:03:34 <zere> urgh... it did that last time too
19:03:43 <apmon_> I have to go now, but I'll read up on what has been discussed
19:04:34 <zere> my personal feeling is that it wouldn't generate *that* much spam... although if it did we'd need to be ready to switch on something like captcha
19:05:00 <zere> #topic AoB
19:05:14 <zere> anyone have anything else they'd like to discuss?
19:05:39 <ppawel> yeah I'd like to briefly discuss OWL but that may be on priv
19:05:53 <zere> sure
19:06:15 <zere> another thing is that europe moves off DST on sunday, i think
19:06:39 <zere> so the next meeting will be different time, either in the US or europe.
19:07:25 <shaunmcdonald> zere: no i think about it ;-)
19:09:07 <zere> ok. any interest one way or the other here, or shall i flip a coin? ;-)
19:09:56 <shaunmcdonald> flip a coin, I don't think it makes much difference, though when did/will the US change?
19:11:41 <jfire> Nov 4
19:11:58 <zere> so it's just the one meeting in the middle, then
19:13:10 <zere> heads it's the same time in the US, tails in europe
19:13:20 <zere> heads
19:14:08 <zere> so that's an hour earlier in europe? 6pm GMT next week
19:14:15 <zere> thanks all for coming!