README.md 17.2 KB
Newer Older
Melvin Keskin's avatar
Melvin Keskin committed
1
2
3
4
5
6
<!--
SPDX-FileCopyrightText: 2021 Melvin Keskin <melvo@olomono.de>

SPDX-License-Identifier: AGPL-3.0-or-later
-->

7
# [XMPP Providers](https://providers.xmpp.net) - Curated List of Providers for Registration and Autocomplete
8

9
10
11
[![XMPP Providers: A](https://invent.kde.org/melvo/xmpp-providers/-/jobs/artifacts/master/raw/badge-count-A.svg?job=badges)](https://invent.kde.org/melvo/xmpp-providers/-/jobs/artifacts/master/raw/providers-A.json?job=filtered-provider-lists)
[![XMPP Providers: B](https://invent.kde.org/melvo/xmpp-providers/-/jobs/artifacts/master/raw/badge-count-B.svg?job=badges)](https://invent.kde.org/melvo/xmpp-providers/-/jobs/artifacts/master/raw/providers-B.json?job=filtered-provider-lists)
[![XMPP Providers: C](https://invent.kde.org/melvo/xmpp-providers/-/jobs/artifacts/master/raw/badge-count-C.svg?job=badges)](https://invent.kde.org/melvo/xmpp-providers/-/jobs/artifacts/master/raw/providers-C.json?job=filtered-provider-lists)
12
[![XMPP Providers: D](https://invent.kde.org/melvo/xmpp-providers/-/jobs/artifacts/master/raw/badge-count-D.svg?job=badges)](https://invent.kde.org/melvo/xmpp-providers/-/jobs/artifacts/master/raw/providers-D.json?job=filtered-provider-lists)
13

Melvin Keskin's avatar
Melvin Keskin committed
14
15
[![REUSE status](https://api.reuse.software/badge/invent.kde.org/melvo/xmpp-providers)](https://api.reuse.software/info/invent.kde.org/melvo/xmpp-providers)

16
This project provides a machine-readable curated list of [XMPP](https://xmpp.org/about/technology-overview/) providers and a script for filtering it.
17
18
The JSON list can be used by XMPP clients or other services for provider suggestions.
Visit the [official website](https://providers.xmpp.net) to see it in action!
19
20
21
22
23

Each **client** has different requirements on a provider: The script in this repository can be used to create a **statically filtered** provider list.

Each **user** has different requirements on a provider: At runtime, the statically filtered list can be **dynamically filtered** on the user's demands.

24
25
26
27
28
29
30
31
32
33
34
35
36
## Main Goals

This project has two main goals:
1. Simplifying the onboarding of new XMPP users by a list of XMPP providers that can be integrated into XMPP clients and used for choosing a provider for registration
1. Improving the providers' features, security, support and documentation by defining high quality standards and providing information to achieve them

## Contributing

Help us to make *free communication* possible and contribute to this project!

**Please read the [contribution guidelines](/CONTRIBUTING.md) carefully before creating a merge request.**
That saves you and the reviewers a lot of time in the review process.

37
38
39
40
41
42
## Categories

The script to create a filtered list can be used for specific requirements.
However, to provide a standardized way of categorizing providers, there are three main options for filtering the list.

The following categories are used to distinguish between different quality levels the checked provders can offer.
43
44
45
Category A and B offer the best, C an average and D the worst user experience during onboarding and further usage.
Therefore, **A and B are the recommended categories for registrations**!
The unfiltered provider list corresponds to category D.
46

47
48
49
50
Category A is a subset of B, B a subset of C and C a subset of D.
Thus, providers of A can be used for the purposes of B, C and D as well.
Providers of B can be used for the purposes of C and D as well.
Providers of C can be used for the purposes of D as well.
51
52
53
54
55
56
57
58
59

### Category A: Automatically Chosen

Providers in this category can be used for an automatic registration.

### Category B: Manually Selectable

Providers in this category can be used for a manual registration.

60
### Category C: Completely Customizable
61

62
63
64
65
66
Providers in this category can be used for completely customized filtering.

### Category D: Autocomplete

Providers in this category can be used for autocomplete.
67

68
## Usage
69

70
If you are an XMPP client developer or would like to create a service based on XMPP Providers, you can create filtered lists on your own or use the daily-updated prefiltered lists.
71

72
73
74
75
The prefiltered lists are versioned and available for all categories (replace `<version>` and `<category>` with the desired values):
```
https://data.xmpp.net/providers/v<version>/providers-<category>.json
```
76

77
Example for a prefiltered list with providers of category **B**:
78
```
79
80
81
82
83
84
85
86
87
88
89
https://data.xmpp.net/providers/v1/providers-B.json
```

Prefiltered **s**imple lists, which contain only the provider domains, are also available:
```
https://data.xmpp.net/providers/v<version>/providers-<category>s.json
```

Example for a prefiltered simple list with providers of category **D**:
```
https://data.xmpp.net/providers/v1/providers-Ds.json
90
91
92
93
94
95
96
97
98
99
```

The version is a natural number and determines breaking changes.
A breaking change means that the format of the lists changes in a way an application parsing it could not handle it anymore.
In that case, the version is incremented and lists of older versions are not updated anymore.
That would be the case if properties are deleted or their names changed but not if a new property is added or a property value changed.

### Client Integration

Here are the recommended steps to integrate the provider lists in an XMPP client:
100
101
1. Download a prefiltered [provider list for category B](https://data.xmpp.net/providers/v1/providers-B.json).
1. Download a prefiltered [simple provider list for category D](https://data.xmpp.net/providers/v1/providers-Ds.json).
102
103
104
105
106
107
108
109
110
111
1. Include both lists in your client's source code / executable.
1. If you have a registration process without user interaction (i.e., automatic registration such as [Kaidan's quick onboarding](https://www.kaidan.im/2020/01/08/Easy-Registration/)), retrieve the providers of category A from the list for category B.
1. For a manual registration, use all providers from the list of category B.
1. Use the list for category D to autocomplete chat addresses after registration.

The provider lists are primarily used to choose a provider that suits the user best.
Its data can be displayed or used to filter suitable providers during the manual registration.
But the provider list for category B can also be used during or after registration for speicifc actions (e.g., to open a provider's website or send a support message).

You should also create a script or use another way to update the integrated provider lists on a regular basis.
112

113
114
## Properties

115
116
117
The providers file `providers.json` is the source to create all filtered provider lists.
The properties in this section are in the providers file and in the provider lists.
There is another property `category` which is only in the provider lists.
118

119
120
121
122
123
The properties of a provider are mapped to its server address / domain / JID (e.g., `example.org`).
In the providers file, the JID is the key of the corresponding provider's JSON object.
Whereas, in the provider lists, the JID is a JSON object within the JSON array's value for the corresponding provider.

### Basic Information
124
125
126

The following properties do **not affect** the provider's **category**:

127
128
Information (Key in JSON File) | Description / Data Type / Unit of Measurement
---|---
129
130
lastCheck | [ISO YYYY-MM-DD date](https://en.wikipedia.org/wiki/ISO_8601#Dates) (e.g., 2021-01-16)
website | mappings from lower-case [two-letter (639-1) language code](https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes) to link of website language version or {} for n/a
131
[busFactor](https://en.wikipedia.org/wiki/Bus_factor) | minimum number of team members that the service could not survive losing or -1 for n/a
132
company | true or false
133
passwordReset | mappings from lower-case [two-letter (639-1) language code](https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes) to link of web page language version used for automatic password reset (e.g., via email) / web page describing how to manually reset password (e.g., by contacting the provider) or {} for n/a
134

135
### Criteria
136
137
138

The table shows the following two circumstances for category A and B:

139
140
A provider is in a specific category if it meets all of the criteria listed in the table below.
A condition can be `true`, `false` or true if a specific case such as greater or lower than a specific value is applicable.
141
142
If the data type is a list or mapping, a condition including numbers corresponds to the count of their elements.
If the data type is a date, a condition including numbers corresponds to the age in days.
143
144

Here is an example for the in-band and web registration:
145
146
*A provider that has no inBandRegistration is not in category A.*
*But if the provider has a registrationWebPage, it is in category B if it also meets all of the other criteria for B.*
147

148
149
The following properties **affect** the provider's **category**:

150
151
152
153
154
155
156
Criterion (Key in JSON File) | Description / Data Type / Unit of Measurement | Category A | Category B | Category C
---|---|---|---|---
[inBandRegistration](https://xmpp.org/extensions/xep-0077.html#usecases-register) | true if registration via XMPP client supported, otherwise false | true | true or registrationWebPage >= 1 | true or registrationWebPage >= 1
registrationWebPage | mappings from lower-case [two-letter (639-1) language code](https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes) to link of web page language version or {} for n/a | >= 1 or inBandRegistration | >= 1 or inBandRegistration | >= 1 or inBandRegistration
rating[XmppComplianceTester](https://compliance.conversations.im) | score (number in percentage) | = 100 | = 100 | >= 90
rating[ImObservatory](https://xmpp.net)ClientToServer | score (upper-case letter) | >= A | >= A | >= A
rating[ImObservatory](https://xmpp.net)ServerToServer | score (upper-case letter) | >= A | >= A | >= A
157
158
159
160
maximum[HttpFileUpload](https://xmpp.org/extensions/xep-0363.html)FileSize | number in megabytes (MB), 0 for no limit or -1 for n/a or less than 1 MB | 0 or >= 20 | 0 or >= 20 | >= 0
maximum[HttpFileUpload](https://xmpp.org/extensions/xep-0363.html)TotalSize | number in megabytes (MB), 0 for no limit or -1 for n/a or less than 1 MB | 0 or >= 100 | 0 or >= 100 | >= 0
maximum[HttpFileUpload](https://xmpp.org/extensions/xep-0363.html)StorageTime | number in days, 0 for no limit or -1 for n/a or less than 1 day | 0 or >= 7 | 0 or >= 7 | >= 0
maximum[MessageArchiveManagement](https://xmpp.org/extensions/xep-0313.html)StorageTime | number in days, 0 for no limit or -1 for n/a or less than 1 day | 0 or >= 7 | 0 or >= 7 | >= 0
161
professionalHosting | true if hosted with good internet connection speed, uninterruptible power supply, access protection and regular backups, otherwise false | true | true | true or false
162
163
164
165
166
167
168
freeOfCharge | true if unpaid service, otherwise false | true | true or false | true or false
legalNotice | mappings from lower-case [two-letter (639-1) language code](https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes) to link of legalNotice language version or {} for n/a | >= 1 | >= 1 | >= 0
serverLocations | list of lower-case [two-letter country codes](https://en.wikipedia.org/wiki/ISO_3166-1_alpha-2) or [] for n/a | >= 1 | >= 1 | >= 0
groupChatSupport | mappings from lower-case [two-letter (639-1) language code](https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes) to list of group chat addresses or {} for n/a | >= 1 or chatSupport >= 1 or emailSupport >= 1 | >= 1 or chatSupport >= 1 or emailSupport >= 1 | >= 1 or chatSupport >= 1 or emailSupport >= 1
chatSupport | mappings from lower-case [two-letter (639-1) language code](https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes) to list of chat addresses or {} for n/a | >= 1 or groupChatSupport >= 1 or emailSupport >= 1 | >= 1 or groupChatSupport >= 1 or emailSupport >= 1 | >= 1 or groupChatSupport >= 1 or emailSupport >= 1
emailSupport | mappings from lower-case [two-letter (639-1) language code](https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes) to list of email addresses or {} for n/a | >= 1 or groupChatSupport >= 1 or chatSupport >= 1 | >= 1 or groupChatSupport >= 1 or chatSupport >= 1 | >= 1 or groupChatSupport >= 1 or chatSupport >= 1
since | [ISO YYYY-MM-DD date](https://en.wikipedia.org/wiki/ISO_8601#Dates) since the provider is available or listed for n/a | > 365 | > 365 | >= 0
169

170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
## Badges

If you are an XMPP provider, you can embed a badge showing your category in your website.
Visit your details page on the [XMPP Providers website](https://providers.xmpp.net).
It contains the code you have to embed.

If you want to provide a service based on XMPP Providers, you can get a badge showing a providers category (replace `example.org` with the desired domain):
```
https://invent.kde.org/melvo/xmpp-providers/-/jobs/artifacts/master/raw/badges/example.org.svg?job=badges
```

You can also fetch all badges together:
```
https://invent.kde.org/melvo/xmpp-providers/-/jobs/artifacts/master/download/?job=badges
```

## Results

If you are an XMPP provider, your details page on the [XMPP Providers website](https://providers.xmpp.net) contains your categorization results.

If you want to provide a service based on XMPP Providers, you can get the categorization results for a provider (replace `example.org` with the desired domain):
```
https://invent.kde.org/melvo/xmpp-providers/-/jobs/artifacts/master/raw/results/example.org.json?job=filtered-provider-lists
```
The JSON file lists all properties that do not meet the criteria for being in a specific category.

You can also fetch all results together:
```
https://invent.kde.org/melvo/xmpp-providers/-/jobs/artifacts/master/download/?job=filtered-provider-lists
```

201
202
## Filter Script

203
The script `filter.py` can be used to create filtered lists of providers (called **provider lists**).
204
205
206

### Usage

207
You can see all options for running the script:
208
```
209
./filter.py -h
210
211
```

212
Provider lists for all categories are created if you run the script without arguments:
213
214
215
216
```
./filter.py
```

217
218
219
If you want to create a filtered list which can be used by a client, simply enter the category name.

Example for creating a list of category **A** providers:
220
```
221
./filter.py -A
222
223
```

224
225
226
227
You can create a provider list containing all providers for completely customized filtering (e.g., by an own filter script or at runtime):
```
./filter.py -C
```
228

229
230
A **s**imple list containing only the domains of the providers is also possible.

231
Example for creating a domain list of category **D** providers to use it only for autocomplete:
232
```
233
./filter.py -s -C
234
```
235

236
237
The **c**ategories of the filtered providers can be included in their entries.

238
Example for creating a list of category **D** providers that includes their best categories.
239
```
240
./filter.py -c -D
241
242
```

243
You can create files containing the **r**esults of the filtering (called **categorization results**).
244
Those files include the providers' properties that do not meet the criteria for being in specific categories.
245
246
247
248
249
250

Example for creating result files for all providers:
```
./filter.py -r
```

251
252
If you are interested in specific providers, you can append them to the command.

253
Example for creating a list of category **B** providers out of *example.org* and *example.com*:
254
255
256
257
```
./filter.py -B example.org example.com
```

258
The script can be run in **d**ebug mode to see why providers are not in a specific category.
259

260
Example for creating a list of category **A** providers and logging additional information:
261
262
263
264
265
266
```
./filter.py -d -A
```

Furthermore, the arguments can be combined to show which criteria specific providers do not meet for being in a specific category.

267
Example for creating a list of category **A** providers out of *example.org* and *example.com* and logging additional information:
268
269
270
```
./filter.py -d -A example.org example.com
```
271
272
273
274

## Badge Script

The script `badge.py` can be used to create badges for the specified categories.
Melvin Keskin's avatar
Melvin Keskin committed
275
276
277
278
279
280
281
282
283
It uses the files created by the filter script.
Thus, the filter script must be run before running the badge script.

Template files are used for generating the badges.
All badges contain the category.
Additionally, it is possible to create badges containing the count of providers in a specific category.

The badge script has two phases:
1. It generates badges for all categories.
284
1. It creates the directory `badges` and fills it with badges for all providers (called **provider badges**).
285
286
287
288
289
290
291
292
293
294
295
296
297

### Usage

You can see all options for running the script:
```
./badge.py -h
```

Badges for all categories are created if you run the script without arguments:
```
./badge.py
```

Melvin Keskin's avatar
Melvin Keskin committed
298
299
300
301
302
The badges for providers can be created as symbolic **l**inks instead of regular files:
```
./badge.py -l
```

303
304
305
306
307
308
309
If you want to create a badge which can be used by a provider, simply enter the category name.

Example for creating a badge for category **A**:
```
./badge.py -A
```

Melvin Keskin's avatar
Melvin Keskin committed
310
Badges containing the **c**ount of providers in specific categories are also possible.
311

Melvin Keskin's avatar
Melvin Keskin committed
312
Example for creating a badge containing the count of providers in category **C** in addition to the normal badges:
313
314
315
316
317
318
319
320
321
322
```
./badge.py -c -C
```

The script can be run in **d**ebug mode to see detailed information about the process.

Example for creating a badge for category **A** and logging additional information:
```
./badge.py -d -A
```
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346

## URL Checker

The script `check_urls.py` can be used to check URLs.
It succeeds if all checked URLs are reachable.

### Usage

You can see all options for running the script:
```
./check_urls.py -h
```

The URLs of all appropriate files are checked if you run the script without arguments:
```
./check_urls.py
```

It is also possible to check only the URLs in specific files.

Example for checking the URLs in `providers.json` and `filter.py`:
```
./check_urls.py providers.json filter.py
```