1,756
edits
User talk:MarkDilley: Difference between revisions
→InterwikiMap
Leucosticte (talk | contribs) |
Leucosticte (talk | contribs) |
||
| Line 281: | Line 281: | ||
Please note that there are two parts to this process of data accumulation and distribution: the sending and receiving parts. Step 1 is to get the wikis to reveal more of their data via [[mw:API:Meta]] modules. Then it will be possible to harvest that data using bots, and push it to WikiIndex pages, which are already organized using templates and therefore would be bot-friendly, if you want to go the bot route. Step 2 is to harvest all of WikiIndex's data and push it into a database table (or, to the extent that SMW is in use, it's already in a database that can be conveniently queried). Step 3 is to allow wikis to pull that data from the central repository, which could be hosted at WikiIndex, or [http://meta.inclumedia.org Meta-Inclu], or wherever else is most convenient and can handle the traffic (viz., 20,000 wikis hitting it perhaps daily; that shouldn't be too big a deal, since it's only about, perhaps, a kilobyte of text about each wiki being transferred; so that adds up to 20,000 kilobytes/request multipled by 20,000 wikis/day, which equals 400,000,000 kilobytes/request-day, or 400 megabytes/request-day; that's not so bad). Actually, I may be lowballing it a bit, though, depending on how many fields of data there are going to be. | Please note that there are two parts to this process of data accumulation and distribution: the sending and receiving parts. Step 1 is to get the wikis to reveal more of their data via [[mw:API:Meta]] modules. Then it will be possible to harvest that data using bots, and push it to WikiIndex pages, which are already organized using templates and therefore would be bot-friendly, if you want to go the bot route. Step 2 is to harvest all of WikiIndex's data and push it into a database table (or, to the extent that SMW is in use, it's already in a database that can be conveniently queried). Step 3 is to allow wikis to pull that data from the central repository, which could be hosted at WikiIndex, or [http://meta.inclumedia.org Meta-Inclu], or wherever else is most convenient and can handle the traffic (viz., 20,000 wikis hitting it perhaps daily; that shouldn't be too big a deal, since it's only about, perhaps, a kilobyte of text about each wiki being transferred; so that adds up to 20,000 kilobytes/request multipled by 20,000 wikis/day, which equals 400,000,000 kilobytes/request-day, or 400 megabytes/request-day; that's not so bad). Actually, I may be lowballing it a bit, though, depending on how many fields of data there are going to be. | ||
Step 4 is for each wiki to filter that data according to its criteria. The question is, will there be a demand for such filtering, or will wikis just want to load all 20,000 prefixes into the interwiki tables? If the latter, then maybe we don't need to distribute all that site data, in which case, the data transfer load can be significantly cut down to something more manageable. In other words, we can reduce it from a kilobyte of text per wiki to maybe one-quarter that. I think, though, that at a minimum they will want to filter based on wiki status. | Step 4 is for each wiki to filter that data according to its criteria. The question is, will there be a demand for such filtering, or will wikis just want to load all 20,000 prefixes into the interwiki tables? If the latter, then maybe we don't need to distribute all that site data, in which case, the data transfer load can be significantly cut down to something more manageable. In other words, we can reduce it from a kilobyte of text per wiki to maybe one-quarter that. I think, though, that at a minimum they will want to filter based on wiki status (e.g. Dead, Spammed, etc.) | ||
I think as a minimum, though, we should have, as an API-accessible field on each wiki using InterwikiMap, a URL for some sort of general information page. Maybe there should be two URLs — one for the wiki's about page (so we get their view) and one for Wikiindex's page (so we get the Wikiindex community's view). Wikiindex, of course, can link to even more articles (e.g. in some cases, Wikipedia articles) giving more views about the wikis in question. But I think it's important that users browsing through the interwiki list have a quick way to get info about the various wikis they see on it. | I think as a minimum, though, we should have, as an API-accessible field on each wiki using InterwikiMap, a URL for some sort of general information page. Maybe there should be two URLs — one for the wiki's about page (so we get their view) and one for Wikiindex's page (so we get the Wikiindex community's view). Wikiindex, of course, can link to even more articles (e.g. in some cases, Wikipedia articles) giving more views about the wikis in question. But I think it's important that users browsing through the interwiki list have a quick way to get info about the various wikis they see on it. | ||
There's a question of what to use for the wiki's about page. Currently, that data hasn't yet been accumulated. [[Template:Wiki]] has a wikinode parameter, but no about page parameter. Should we conclude that the wikinode idea is moribund, or should we keep pushing for its adoption? In the meantime, should we begin accumulating URLs for "about" pages? Perhaps in some cases, the about page ''would'' be the wikinode, but usually not. [[User:Leucosticte|Leucosticte]] ([[User talk:Leucosticte|talk]]) 19:36, 20 November 2012 (PST) | There's a question of what to use for the wiki's about page. Currently, that data hasn't yet been accumulated. [[Template:Wiki]] has a wikinode parameter, but no about page parameter. Should we conclude that the wikinode idea is moribund, or should we keep pushing for its adoption? In the meantime, should we begin accumulating URLs for "about" pages? Perhaps in some cases, the about page ''would'' be the wikinode, but usually not. [[User:Leucosticte|Leucosticte]] ([[User talk:Leucosticte|talk]]) 19:36, 20 November 2012 (PST) | ||