OpenStreetMap Foundation, Operations Meeting* - Agenda & Minutes
Wednesday September 9th 2020, 18:00 London time
Location: Video room at https://osmvideo.cloud68.co
* Please note that this was not strictly an OWG meeting.
- 1 Participants
- 2 Administrative
- 3 Fastly status
- 4 AWS
- 5 AMS hardware
- 6 UCL
- 7 Ironbelly replacement
- 8 Open Ops Tickets
- 9 Next meeting
- Grant Slater (OWG)
- Tom Hughes (OWG)
- Paul Norman (OWG, board)
- Guillaume Rischard (board)
- Allan Mustard (board)
- Michal Migurski
Minutes by Dorothea.
Reportage and action item updates
2020-08-26 Tom to look at road ahead for OAuth. [Topic: Merge forums, OSQA, MLs to discourse?] https://github.com/openstreetmap/openstreetmap-website/issues/1408
Did some investigation - branch with some code. Better understanding of OAuth 2 and options. Doable.
- Get OAuth 2 working on different set of endpoints.
- See if it's possible perhaps to merge the cloud application table.
Action item: Tom to update ticket https://github.com/openstreetmap/openstreetmap-website/issues/1408
Action items from this meeting
- Tom to update OAuth ticket https://github.com/openstreetmap/openstreetmap-website/issues/1408 [2020-09-09 Reportage, related to 2020-08-26 action item]
- [Topic: Fastly status] Paul to draft email saying we are looking at the privacy issue. The plan is to switch and 3rd parties will have access to logs and traffic data of CDN nodes. Could include link to Fastly terms.
- [not assigned] [Topic: Fastly status] Email the board that we're giving access to logs for exports and we plan to do it for broader users of OSM and 3rd party websites, unless we hear otherwise.
- [not assigned] [Topic: AWS] Speak to AWS person about going ahead with open data program with official OSM S3 bucket.
- [not assigned] [Topic: AWS] Decide on services we need to run on AWS. Need clearance.
- [not assigned] [Topic: AWS] Work out rough budget.
- [not assigned] [Topic: AWS] Talk to OpenAerial Map/HOT.
- [not assigned] [Topic: Federating OSM communities' rooms through OSMF-hosted Matrix servers] Evaluate effort required. Constrain the scope to what we can support and perhaps ask volunteers to step in.
- [Topic: Ironbelly replacement] Paul to work out a proposal for the Ironbelly replacement.
Status update on Fastly test
- Issue raised: Privacy aspect/TOU.
- We already have a node in US.
- Blocker for doing more testing.
- Paul to draft email saying we are looking at this. The plan is to switch and 3rd parties will have access to logs and traffic data of CDN nodes. Could include link to Fastly terms.
- Email the board that we're doing it for exports and we plan to do it for broader users of OSM and 3rd party websites, unless we hear otherwise.
Next steps Decide which countries to send. Set time frame short before test expansion.
# Status update on how asking Fastly support about logging went
- Question was about to move our blog processing which generates public usage details.
- We haven't asked them yet. Could ask Ian.
Grant had a call with AWS person about free credits
1. Planet distribution can be sponsored under the open source open data program.
- They can give us account with full admin access and they will pay.
- Any service has to be related with data generation/distribution - can't run tile-serving on that account.
2. Using a credit program, they are potentially willing to fund us with credits that may go all the way up to as much as we use.
- Based on actual monthly cost.
- Difficult to use this on long term basis as it is based on renewals but the logistics team might work something out (they use OSM extensively).
- They suggested that we speak to OpenAerial Map or HOT as they already receive sponsorship for doing that.
- We could potentially get some imagery "exclusive" to us.
- Speak to AWS person about going ahead with open data program with official OSM S3 bucket
- Work out rough budgets.
- Talk to OpenAerial Map/HOT.
- Decide on services we need to run. Need clearance.
- Migrate things and come up with a base that's easy to manage.
- Serve primary copy of planet file on S3 as part of creation process and encourage people who use AWS to get the planet and diffs from there
- We could reduce the amount of Planet files that we store locally on our planet mirror and redirect people across.
- Wouldn't cache planet file.
- In parallel to Fastly.
- Has download integrity.
- Used for distribution of Ubuntu ISO images and by gaming companies, such as Blizzard.
- Useful for people with poor internet connections.
Old servers: Dulcy (cpu from 2014), Ironbelly and Thorns (G7)
GPX traces, planet dumps, replication files.
Previous discussion about stop using back-ends and use just front-ends.
- disable thorns
- keep nominatim server as-is for now
- Rammoth (motherboard issue fixed, trips the power when turned on)
- 3 G5 powered down, not wiped yet.
Guillaume offered to drop by.
Use data destruction company in AMS.
Under Covid access restrictions.
Suggestion: If it's painful to get access again, put hardware elsewhere.
- Depends on what we'll do with S3.
- Get pizza box to store a couple of planet files.
- Database backup osm-2020-09-07.dmp - 580GB (it's compressed).
- Serve only latest planet data - anything older serve via S3 (back-up scripts may only have permission to upload)
Action item: Paul to work out a proposal for the Ironbelly replacement.
Open Ops Tickets
Review open, what needs policy and what needs someone to help with.
Create .torrent files as part of planet creation
- Trackers: Will stay away from those associated with piracy.
- RSS: not an initial feature. Maybe later.
Federating OSM communities' rooms through OSMF-hosted Matrix servers
Database: PostgreSQL probably
Media storage: can use S3
Installation: * Primary method docker based.
* Synapse package also available.
* Probably need volunteers that can write chef.
* Appoint people to manage channels.
Points to consider
- Scale of hardware required.
- Frequency of software updates.
- Any special considerations in upgrading, managing and support.
- Number of bridges people will ask for.
- Effort required.
Hardware: Use thorns (currently not externally visible) instead of (AMS powered down) spikes.
- Ask Frederik, who is running his own instance, about the amount of work expected.
- Start with IRC and simple federation.
- Ideal: OAuth using OSM accounts.
- Matrix do not seem to have a federated authenticated solution.
- Using <OSM_username>@openstreetmap.org might be problematic.
- Evaluate effort required. Constrain the scope to what we can support and perhaps ask volunteers to step in.
- Paul to write to ticket https://github.com/openstreetmap/operations/issues/380#issuecomment-680780871
Wednesday, 23rd of September 2020, 18:00 London time
- Operations meetings are currently being held every 2 Wednesdays, at 18:00 London time.
- Online calendar showing the OPS meetings.