OSMF updates/2020-08/Operations Overview

From OpenStreetMap Foundation

Contains news from Operations Working Group reports and Operations meetings.

2020 budget request

The OWG’s 2020 budget request was approved by the OSMF board. Planned new work includes:

  • refreshing API DB servers;
  • refreshing HP DL360 G6 machines at UCL and OSUOSL;
  • upgrading Nominatim capacity; and
  • using a consultant to make chef cookbooks more testable and make it easier fr others to contribute.

Policies

  • The OWG has approved a new banner policy for event banners on OpenStreetMap.org.
  • The tile usage policy has been amended to reflect current practice and the increased load on the tile infrastructure.

Services

irc.openstreetmap.org

The CGI:IRC page has been upgraded to redirect to the official OFTC webchat IRC. You can continue to use irc.openstreetmap.org, use https://webchat.oftc.net/, or use your choice of IRC client.

nominatim.openstreetmap.org moved to nginx for web serving

nginx allows us to handle additional traffic and simplifies the quality of service management to fairly serve users following our usage policy.

planet.openstreetmap.org

Downloads from planet.openstreetmap.org will now use the a mirror when downloading PBF files. Bzip2 files already used the mirror.

We will soon be adding BitTorrent as another way to download the weekly Planet.osm exports. Please test the EXPERIMENTAL torrent files here: https://osm.cquest.org/torrents/

svn.openstreetmap.org

Is now closed to new accounts and will be made read-only before the end of August 2020. Many relevant http://trac.openstreetmap.org tickets have already moved to https://github.com/openstreetmap/openstreetmap-website/issues We're available to help developers move their project.

trac.openstreetmap.org turned off

The time has come for us to retire our trac bug tracker, as it is becoming a burden for us to support and most projects have moved onto better alternatives.

tile.openstreetmap.org

A number of default user-agents and heavy use apps have had their access to tile.openstreetmap.org restricted. If you are developing software using the OSMF tile servers, be sure to follow the tile usage policy.

Do you run a website or app which uses http://tile.openstreetmap.org? Make sure to switch to the HTTPS tile url. The http:// url has been deprecated.

Tile servers are upgraded to OpenStreetMap Carto 5.2.0.

Tile caches

New caches

Many new tile caches were setup

Cache Location Host
stormfly-01 Corvallis, Oregon OSUOSL, hardware provided by OSMF
gorwen Bogotá, Colombia edgeuno
neak France faimaison
meraxes Paris, Fance Scaleway
naga Paris, France MilkyWan
fuchur São Paulo, Brasil edgeuno
idris Buenos Aires, Argentina edgeuno
glaedr Bogotá, Colombia edgeuno
albi Paris, France Scaleway
Gackelchen Bissen, Luxembourg Gandi
Kokosnuss Germany Strato, thanks to Max Michels.
Falkor Prague, Czech Republic Vodafone
Shruikan Dronten, Netherlands. greenminihost

Also https://blog.openstreetmap.org/2020/04/ but has 2019 data and incomplete.

Would you like to host a tile cache?

If you operate an internet exchange, host company, or otherwise have a site with good internet connectivity and high regional bandwidth, you can look at the tile CDN node requirements. We welcome hosting of tile caches elsewhere, and are particularly looking for tile caches in Africa and Asia. If you are interested, please contact us.

www.openstreetmap.org

Call for featured layers on www.openstreetmap.org

Submissions for featured layers are welcome and will be evaluated when they come in.

New featured layer for public transport

We have approved ÖPNVKarte as a new featured layer on OpenStreetMap.org. blog post.

API

OSM JSON format is now available in production. Supported endpoints are listed on the OSM API 0.6 wiki page.

Four additional API 0.6 endpoints are now being directed to CGImap.

The new database servers, snap-01 and snap-02 have been racked and connected.

Other

Network congestion control

BBR network congestion control was tried, then rolled back after problems.

Upgrade of cgimap

cgimap was upgraded from 0.7.5 to version 0.8.1. JSON output is enabled by default now.

See release notes for a list of all new features/bug fixes.

Hardware

New API database servers have arrived in Amsterdam and the UK. The UK server is awaiting a trip to UCL which needs to be scheduled with COVID-19 restrictions.

Two new servers to replace stormfly-01 and stormfly-02 have arrived in OSUOSL but are awaiting a trip to the data center which needs to be scheduled with COVID-19 restrictions.

4 1U servers were removed from UCL for disposal.

Several failed or failing disks were replaced in machines.

Two servers to replace stormfly-01 and stormfly-02 have been installed at OSUOSL. One is planned to be a tile cache.

Git

The network issues between UCL and Github were diagnosed as a faulty UCL firewall and fixed.

DNS

Our domain registrar, Gandi, is now supporting OpenStreetMap with free registration.

We have switched to using https://stackexchange.github.io/dnscontrol/ to managing all OpenStreetMap domains DNS. https://github.com/openstreetmap/dns/blob/master/dnsconfig.js We also have #DNSSEC enabled on open-maps.org, other domains to follow.

osm.org outbound mails

All OpenStreetMap.org outbound mails are now automatically DKIM signed to confirm sending server authenticity. Additionally SPF IP listed.

Mailing lists

The OpenStreetMap mailing lists now have DMARC "Munge From" enabled by default. https://wiki.list.org/DEV/DMARC

General

Some services have been upgraded to Ubuntu 20.04.

Membership

Radoje Stojisic and Hrvoje Bogner joined the OWG.

If you are interested in joining, please read our membership policies and contact us.

Discussions during operations meetings

In April online Operations meetings with guests were started. Minutes or notes are added at the Working Group minutes page. Discussion topics are presented below.

Also mentioned above

Turn off Trac

https://github.com/openstreetmap/operations/issues/416

See above.

Notes from meeting: Jun

SVN

See above.

Notes from meeting: May

Featured layers call

See above.

Notes from meeting: May

[Closed ticket] Fixing the failing CI tests

The network issues between UCL and Github were diagnosed as a faulty UCL firewall and fixed.

Minutes/notes from meetings: Apr, May 04

[Ticket] Migrate help.openstreetmap.org from OSQA

Github ticket: Migrate help.openstreetmap.org from OSQA #149

help.openstreetmap.org is currently using OSQA (Open Source Question and Answer), which:

  • has been inactive upstream for long time.
  • is on Python/Python2, which goes away.
  • is on a machine that (in 2019 Jan was) running 14.04, which reached End of Life in April 2019.

Options for migration suggested so far include, but are not limited to:

Current status on archiving

Minutes/notes from meetings: Apr, May, Jun (archiving help.osm.org), Jul (archiving help.osm.org)

[Ticket] OSM Forum (FluxBB) update

Related to ticket: Github: OSM Forum (FluxBB) update #439

Notes from meeting: Jul (FluxBB is old and slow)

[Ticket] Adding API key support for tile.osm.org?

Github ticket : Adding API key support for tile.osm.org? #342

Notes from meeting: Jun

[Ticket] Monitoring and alerting system

Related to Github: Switch to Prometheus for monitoring and alerting #360

We are hitting the limits of Munin. Prometheus was suggested as an alternative but is a lot of work to get the same monitoring and alerting coverage we have right now.

Minutes/notes from meetings: May

[Ticket] Wiki reCAPTCHA issue

Github issue: Wiki asks for confirmation code before saving changes without sending one #449

hCaptcha has been suggested as a reCAPTCHA replacement and support for it is coming to mediawiki. Previous blocker: Mediawiki 1.35 being released in August.

Please note that we don't have the resources to develop a custom replacement for reCaptcha. If someone else develops a custom replacement with an OpenStreetMap focus, we would be interested.

Notes from meeting: Jul

US servers packet loss

Notes from meeting: Jul

Machines running old software

planet-dump caused outage

Notes from meeting: Jul

High Availability / Redundancy of OpenStreetMap.org (and primary services)

We have two main data centers for osm.org: one in Netherlands and a warm/coldish copy in the UK.

Minutes/notes from meetings: Apr

New Data Centre Space?

Even if we move stuff to virtual providers and hosting companies, we think we still want to maintain a physical presence somewhere as a second site. Current situation in Slough works, but not for next 5 years.

Minutes/notes from meetings: Apr, Jul (Long term plans)

AWS Config Using Code (Terraform?)

Notes from meeting: Jul

Tile CDN

Tech

Squid / nginx / GeoDNS? Is the software right? Issues and improvement.
What experiments should we be doing to improve the network.

Notes from meeting: May

Tile CDN usage policy (OSM, Friends, Others?)

https://operations.osmfoundation.org/policies/tiles/
Can we continue to be be open to the world, what policies (not yet actions) can we put in place now to encourage people to move to other providers? eg: Do we continue to allow Commercial Sites to use tile.openstreetmap.org ?

Minutes from meeting: Apr

Commercial CDN for Bulk Tile Users

We are considering of using a commercial CDN for bulk tile users. Fastly has expressed interest in supporting OSM via an Open source grant and they are interested in "co-branding" (blogpost). Please see the notes linked below.

Minutes/notes from meetings: Apr, May, Jul 01, Jul 15, Jul 29

Hardware

  • Equinix servers
  • UK replacements
  • Status on the discs to Oregon, to be added to remote management network

Ironbelly

OSMF Servers Info: ironbelly.openstreetmap.org

Notes from meeting: Jul

Imagery Server Future?

OSMF Servers Info:

We'll upgrade Errol.

Notes from meetings: Jul 15 (Replacement of Errol/Kessie), Jul 29

Revision of acceptable use policy to reduce incoming comms

Notes from meeting: Jul

Retrieved from "https://osmfoundation.org/w/index.php?title=OSMF_updates/2020-08/Operations_Overview&oldid=8854"