OSMF updates/2021-08/Operations Overview

From OpenStreetMap Foundation

Operations Working Group (OWG), Contact/Join

Ensuring the operation and smooth running of the XML editing API and data distribution services.

https://operations.osmfoundation.org

Github: OSM operations, OSM website and other repositories.

Below are topics as mentioned in Operations reports at the time of their publication. Please note that depending on when the reports were released, the information might have changed. Some points might appear in two sections.

Acronyms

New data center in Dublin

The OWG has decided to move a site UCL/Bytemark in the UK to Dublin, Ireland. Equinix DB4 was selected as the new site location, Hurricane Electric as the ISP. Like the existing site, the new one will have the resources to host the website if the Amsterdam site is down. All required hardware has been ordered to meet this requirement.

COVID and global supply chain shortages have resulted in extended lead times for purchases, so some hardware has not yet arrived.

2021-09 update: After months of planning, we now have server infrastructure in Dublin Ireland.

Hardware

All new supermicro servers in the 2021 budget have been ordered. Some were scheduled for 2021Q4 and 2022Q1 purchases but were ordered sooner because limited component supply means the lead time could be up to 12 weeks.

In addition to Dublin setup and planned replacements, there are new servers for expanding the capacity of Nominatim (Geocoding service) and the Standard Layer. The geocoding expansion will be in Dublin, and Standard Layer in Dublin and Umeå, Sweden.

New planet servers have been ordered. Using an object store was considered to reduce server requirements, but a large disk server was decided on as the object store route would take an unpredictable amount of time to develop.

Two old HP servers in the US have been donated to OSGeo instead of scrapping them.

Development server

Services

Standard tile layer

The Standard tile layer is the default tile layer on the OpenStreetMap homepage. It is often referred to by the name of the stylesheet OpenStreetMap Carto (OSM Carto or osm-carto for short).

The standard tile layer has switched to using Fastly as the content delivery network rather than a set of donated machines around the world. This drastically simplifies administration and improves service in areas of the world that were underserved before.

In order to reduce mass tile downloading in violation of the tile usage policy, automatic rate limiting by IP has been implemented. This should not impact users following the usage policy but will limit the load a single abusive user can put on the system.

Two new rendering servers have been ordered. One is being installed in Dublin, the other in Umeå, Sweden. Two new rendering servers are running on donated hardware from Appliwave and AARNet. The AARNet server was previously a CDN server.

Planet

The planet distribution service provides planet files, replication diffs, changeset dumps, and other data dumps.

Torrent files are now available for the planet files. They are automatically created and use webseed to distribute the load among the mirrors.

There is now official documentation of the state.txt format.

2021-02 The minutely replication diffs have switched from being produced by Osmosis to being produced by osmdbt.

New Services

TODO: Add Discourse

Monitoring

The OWG has been testing Prometheus for monitoring as a replacement for Munin. We are currently experimenting with long-term storage of metrics with TimescaleDB and Promscale.

Budget

2021-03 The board has approved the OWG budget at the “high” option suggested in February.

OTRS - Finding alternatives to OTRS 6

Testing Discourse to replace some OSMF-hosted communication channels

TLSv1 and TLSv1.1 no longer supported on OSMF run websites

CyclOSM added as a new featured layer on www.openstreetmap.org

Operations meetings every two weeks