Operations/Minutes/2021-05-05
OpenStreetMap Foundation, Operations Meeting* - Agenda & draft minutes
Wednesday 5 May 2021 18:00 London time
Location: Video room at https://osmvideo.cloud68.co
* Please note that this was not strictly an OWG meeting.
Participants
Present:
- Tom Hughes (OWG)
- Grant Slater (OWG)
- Paul Norman (OWG)
- Michal Migurski
- Scott
Minutes by Dorothea Kazazi.
Apologies:
Administrative
Previous minutes
Action items from this meeting
- Grant to check/fix GroupTweet for osm_tech Twitter account.
- OWG to email Toby and suggest chatting to MapTiler. [Topic: Wikimedia]
- Paul to do a circular for ISP in a couple of days. [Topic: Dublin updates - Network requirements change]
- Grant to provide switches model and vendor to Paul. [Topic: Dublin updates - Someone to handle network purchasing]
Reportage
Fastly
| Background |
|---|
|
OSMF currently uses the Fastly content delivery network as cache for tiles in some regions. Fastly related previous discussions: Commercial CDN discussions: |
Communications:
They have a draft that they will run by us and will turn to a blogpost. We are now in contact with Hannah Aubry regarding communications.
Listing as top 3 donors:
Michal suggested waiting, as he wants to have a phone call with them first.
Timescale DB
- Upgraded.
- Crash in US. - Disk failure. See topic: "SSD Disk Failing in US"
Wikimedia
| Background |
|---|
|
Related discussion: Wikimedia challenges with Tile CDN delivery Topic carried over from previous meeting. Related to past action item: 2020-09-23 Toby Negrin (Wikimedia) to ask Wikimedia whether they would be interested in OSMF running a tile service available to Wikipedia and if they would be willing to share hardware resources or expertise. |
- They have an established Content Delivery Network (CDN).
- Rendering back-end recently failed-over (Mapnik based).
- Traffic unknown.
- New style.
- Time-cost would probably be the problem.
- What are the requirements.
Decision: park the issue.
Action: OWG to email Toby and suggest chatting to MapTiler.
Scott introduction and brief on-boarding
Interested in helping in operations/security.
- Background:
- supported Intel's manufacturing infrastructure and applications ~ 5 years.
- managed Debian boxes.
Brief on-boarding
- Communication channels: #osmf-operations and #osm-dev on irc.oftc.net
- Suggestion to get access to one service he's interested in.
- All config managed via Chef.
- Some automated testing on pull requests for Chef, thanks to Michal (Some Gitlab-related failures).
- Service classification policy.
- New hardware for primary services
- second-hand for secondary and tertiary services (use more resources/disk space). Redundancy by adding hardware that can have failures.
- Amazon Web Services (AWS): use for storage and Athena for log analysis.
Grant: Prefers HP hardware, easy to get.
New data center (Dublin) updates
Network requirements change
- Paul changed the network requirements.
- Budget ~ 1K/month
Amsterdam ~250/month
- Tier 1
- Good offer 10 GB link with 10K commit
- Peering disputes with Cogent (used at Amsterdam)
- Would give us a small subnet /24 to /27 IPv4 (~32 IPs minus 5 for their equipment - similar to what we had in Amsterdam). Or could do BGP with them with our own subnet (would need to join RIPE registry).
- Tier 1.
- Offering 2 ports.
- Cheap.
- Tier 2.
- Own fiber over Europe, including Dublin and Amsterdam.
Irish academic network
- Reached through OpenStreetMap Ireland
- We would qualify to peer with them but unknown if we would qualify to send all our traffic through them.
Cross-connect
- ~ 150/month for all
Preferences Paul: Hurricane Electric. Grant: Hurricane Electric.
Decision: Will wait a couple of days.
Action: Paul to do a circular for ISP in a couple of days .
On us doing BGP
- Quotes above do not include us doing our own BGP.
- 2K/year not a big cost
- We would need to manage it.
- With 1 peer - simple, fairly static announcement (minimum: /24)
- RIPE registry: For every member they can allocate a single /24 IPv4 subnet - request then added to waiting list (which is 0 at the moment).
Someone to handle network purchasing
- Switches we have don't do BGP natively.
- We don't need to do BGP on our switches, we can run it on a separate instance and do the announcements across.
- ~2K GBP/each (500-5000 range).
- Cisco SG550X-48 are available.
- Need to buy genuine modules ~300 GBP/each (had problems with thermostats on other ones).
- Hardware vendor in UK that has center in Amsterdam.
Action: Grant to provide switches model and vendor to Paul.
Hot spares in new servers
- In the past hot spares have been put into used, instead of being kept as spares.
- 6K GBP for hot spares in all machines (~10 machines).
- 2 disk redundancy in a raid 6 array or hot spare.
- Worst case scenario: data loss, which also results in loss of time.
Alternative: on-site spare.
Decision: hot spares.
Lead times on hardware purchases
>6 weeks, due to RAM shortages.
Discourse
| Related discussions |
|---|
No updates.
Fastly
In contact with them.
Paul will do a presentation on Mapping US and State of the Map conferences about tile usage.
Some points related to logging not minuted after request.
Next meeting
Wednesday 26 May 2021 18:00 London time
- Operations meetings are currently being held every 2 Wednesdays, at 18:00 London time.
- Online calendar showing the OPS meetings.