Wikivoyage talk:Script policy

So, this page is kind of preemptive. It's not like we have tons of people interested in writing scripts to run against Wikivoyage. The point here was to avoid having a lot of hassle when someone does want to do that.

There are a lot of hoops to jump through to make a script work on Wikivoyage on this page. Probably most of them are unnecessary. It seems a little ungrateful to put all these barriers up to keep people from contributing to Wikivoyage. But the possible downside of having a well-intentioned but bad script run against Wikivoyage -- say, blanking all pages -- seems to outweigh the unfairness.

Comments very welcome. As with (almost all) policies on Wikivoyage, this one is open to editing, discussion, revisions, whatever. Making policy is a community decision; I'm just trying to get the ball rolling. --(WT-en) Evan 18:20, 9 Jan 2004 (EST)

I'd like to suggest that the source code of scripts should be made public or at the very least given to the admins/Evan. This would avoid annoying situations where a useful script is created, but the creator goes away and now nobody else can run it (cf. User:(WT-en) InterLangBot). It should also help in reverting/fixing if the script goes astray. (WT-en) Jpatokal 05:42, 21 Jun 2005 (EDT)

Y'know, one thing missing from the script policy is the definition of a "script". The wording is obviously intended towards scripts that go around editing existing pages, but should scripts that upload completely new data (eg. ja:User:(WV-ja) Kunibotto) also be included? For these, the main safeguard should be that existing content is not overwritten... (WT-en) Jpatokal 12:08, 15 Jul 2005 (EDT)

There's a definition in the first sentence: programs that read and write Wikivoyage pages. Although this includes browser-based editing by humans, and Wik, there's an implication that scripts work without human intervention.

And, no, there was never any intention that the words only applied to existing pages. --(WT-en) Evan 13:26, 15 Jul 2005 (EDT)

At the risk of sounding a little facetious, the browser I'm typing this in is "a program that reads and writes Wikivoyage pages". I also feel it's a bit of a stretch to label a read-only tool like wget as a "script" and to attempt to insist on the full script policy for these (good luck getting anybody to comply). Howzabout programs that modify or create Wikivoyage pages automatically? FWIW, Wikipedia uses the definition automatic processes interacting with Wikipedia over the WWW'. (WT-en) Jpatokal 21:58, 15 Jul 2005 (EDT)

I think you're right that the main type of scripts we're concerned with are those that modify the site. Unruly read-only programs (like wget) are a serious problem and I block an IP at the Apache level about once or twice a month. But I don't think they need to undergo the same level of scrutiny as scripts that modify the site.

I'll try to change the text of this page to make that point. --(WT-en) Evan 16:28, 17 Jul 2005 (EDT)

Lowering the edit throttle rate

So, I've just changed the reasoning on the edit throttle rate (once per minute), which I think was inaccurate before. Our infrastructure is pretty robust right now, so the real reason to throttle edits is to limit the damage a script can do before someone blocks it or sets its run page to "no". Since the community is much bigger, and there are much more active experience users and admins than in the Olden Days when this policy was written, I think we could handle an edit once every 10 seconds... does this sound reasonable? --(WT-en) Evan 11:17, 13 November 2006 (EST)

New discussion (2012)

In 2006, above, Evan proposed lowering the limit to 10 seconds per edit, which I think is more than reasonable. One minute per edit seems to almost defeat the point of bots. However, I don't think any bot should be operating without a bot permission bit, no matter how slowly it goes. LtPowers (talk) 18:35, 6 December 2012 (UTC)

Bot

Swept in from the pub

we need a interwiki bot waiting for wikidata 82.48.137.202 15:46, 17 November 2012 (UTC)

Script policy

Swept in from the pub

Hello all, I am sorry to say but this page is very much outdated: Wikivoyage:Script policy. Bots are mend for making repetitive edits that would be extremely tedious to do manually. I can do 12 edits a minute only by hand, so I don't understand at all why the number of edits a minute is on one a minute... The whole page gives me an impression of fear for bots/scripts instead of a realistic approach to bots and the page itself has its origin years ago when bots weren't that developed and advanced. About the "problems" in the section pros and coins:

• Unintended consequences: I would expect that you let someone do test edits to gain trust before giving someone a botbit. Afterwards the botowner is responsible for the edits, and must fix the problems on the articles first before continuing.
• Unilateral: than the bot/script isn't written well.
• Suck up resources: if something needs to be changed on a thousand pages which can be done by bot, the bot will need less time and less resources than if a user would do it by hand. This results in that users spend much time in things that could have been done by a bot and the same time used for writing articles by the user. Also if a community wants something, this is more important than worrying about resources. Within Wikimedia the resources-argument should not been taken into account. (Read w:Wikipedia:Don't worry about performance)
• Unnecessary: one valuable point, but does not rule every bot/script out.
• No new knowledge: in general true, but sometimes the eye of the visitor/editor also wants a nice page to look at instead of a total mess.

About the section Script policy:

• 5. I use AutoWikiBrowser (AWB) and I see no way how that would ever be possible. Also for other scripts and bots I don't see how they would do that. But in AWB the bot stops when edited the talk page of the user.
• 6. One change a minute? Sorry, but this doesn't sound reasonable at all. If a bot isn't approved and doesn't has his botbit, sure this can be used to prevent spamming the recent changes and prevent problems. But when the bot is approved, this doesn't make sense. When a bot is approved, the approval must say: we trust you and you may go ahead (of course with respect of the guidelines/policies). If after approval the bot does untrusted things and keeps doing the same after talking to the owner, the botbit should be taken away. Also one edit a minute would make using bots/scripts very uneasy. I personally use AWB, a piece of software in what I can set some parameters to perform a task. Depending on the task, I can run it automatically (I myself check in the mean while some samples to make sure the bot works properly.) or semi-automatically with what I need to hit the save button for each edit after checking the proposed changes. In two seconds a user can see if the edit is right and saves. Why then waiting 58 seconds for the next edit to be made? I can do 12 edits a minute just by hand alone.
• 7. Really outdated.

My background: I often work with bots to help other users which requests for help because they got too tired with doing all by hand, while the bot is faster, less errors and much less tedious. I help everywhere I can help, so that other users can continue with writing articles and adding content.
In the past weeks I got such request from User:JamesA, I did some test edits and got the bot approved by a bureaucrat. In this way I hope to help this project to be fixed and cleaned up to complete the transfer to Wikimedia, so that the main goal will be the articles again. Due personal circumstances I wasn't able to perform the bot yet and today I found by accident the page Wikivoyage:Script policy. I do want to respect the policies of a project, but trusting on the request of the users who asked and approved the bot, I am planning to do the requested work which will be in conflict with the script policy. If someone has serious problems with that, please write on my talk page. But the main request here is: please change the outdated script policy! Greetings - Romaine (talk) 16:14, 27 November 2012 (UTC)

This is an issue that could have been mentioned at #Which policies need new discussion? above. I'd say one of the big benefits of moving to the WMF is access to more & better bot scripts and the policy needs a thorough revision. Pashley (talk) 18:07, 27 November 2012 (UTC)

The discussion above at #Geocoding bot was moved to Wikivoyage:Script_nominations. Should your bot go there?

Is there a list somewhere of WP or global WMF bots we could look at to see which would work here? There must be, and the location is probably obvious to some, but it is not obvious to me as a WT admin just adjusting to being here. Pashley (talk) 18:29, 27 November 2012 (UTC)

This policy was written in 2005 for a different site. Romaine, if you want to take a stab at rewriting/updating it, please plunge forward, and others will revert anything contentious. --Peter ^Talk 19:32, 27 November 2012 (UTC)

I can give it a try, but such should be adopted by a community. Romaine (talk) 21:08, 27 November 2012 (UTC)

If no one objects, it's safe to assume your changes have been adopted ;) --Peter ^Talk 21:31, 27 November 2012 (UTC)

A list of en.Wikipedia bots is w:Wikipedia:Bots/Status although the description of each is rather vague and general. There's an entire wiki (botwiki.sno.cc) dedicated to information on how to write 'bots using existing frameworks such as pywikipediabot. The {{geo}} import from WP (Wikivoyage:Script nominations#The Anomebot2) looks like a useful application for a 'bot. Odd no one has commented there. (I'd prefer to see the co-ordinates in the page here instead of relying on downstream third-party apps like WikiSherpa to try to piece this together from cumbersome multiple sources. We do want all of the info on one city in one place, generally.) K7L (talk) 20:25, 27 November 2012 (UTC)

(after bwc) Perhaps I am a little bit confused with all pages mentioned.
A list of bots is a bit difficult to answer. I don't understand what you mean exactly, which is caused by the multi ways the word 'bot' is used. A bot is a piece of software operated by a user, which uses a botaccount (mostly bot with botbit) to edit. Many users let their bot run continuously (cronjobs) and often use for that purpose our Toolserver, but the continuous running is mostly for bots who update interwikis, archive bot, Commonsdelinker, and such. (Commonsdelinker is a bot which removes a file from a page on a local wiki when the file is deleted on Commons, or when a file is renamed/exact duplicate this is done on the local wiki as well. This bot normally will show up in recent changes so that users can possibly add another file instead.) Other bots are used for specific single tasks, and then a bot can be used automatically or semi-automatically (to allow the user to check and adjust if needed). The software used for a bot can be various, but the most used software is Pywikipediabot and AutoWikiBrowser (AWB). On meta there is a general bot policy, but only is effected in those wiki's which have adopted it. The general bot policy is not always that accurate, but if it is taken up a bit freely it works well. (Like the edit throttle and peak hours are mostly mend for cronjobs and not for bots with individual tasks, also very difficult to apply with (semi-)automatic AWB. And peak hours depend strongly on which language version.) Romaine (talk) 21:08, 27 November 2012 (UTC)

Hello! Romaine pointed me to this discussion after I approached him regarding his (excellent) bot work on sv:, and suggested a bot approval would be needed. I do miss Wikivoyage Shared, because it was an excellent site for such interlanguage cooperation - we simply cannot keep track on what's happening on all the different language versions. Having said that, I am not convinced we should just bypass the script policy which, after all, has worked fine so far, even if it surely needs updating. I suggest Romaine lists his suggested changes on the policy talk page, and I am sure there won't be much resistance, since we all want to build an excellent site. I know for sure that sv: needs bot help. Perhaps we should just adopt this policy? Riggwelter (talk) 08:30, 28 November 2012 (UTC)

Wikipedia's needs are much (much much) different from ours, at least at the moment. I would not, for instance, support the formation of a Bot Approvals Group here on Wikivoyage. LtPowers (talk) 16:48, 28 November 2012 (UTC)

"Shared" is now at

• meta:Wikivoyage—coordination across Wikivoyage language versions, plus
• Commons:—shared file repository

--Rogerhc (talk) 17:25, 28 November 2012 (UTC)

Changed! Romaine (talk) 20:51, 5 December 2012 (UTC)

A few tasks

Swept in from the pub

As I used to do on Wikitravel, I started running a few syntax checks, here are the detected errors that need to be fixed. I will fix them when I have time, but if someone wants to fix them please go ahead (please remove the items you fixed) Thanks! :-) Nicolas1981 (talk) 04:08, 4 December 2012 (UTC)

Ad hoc admin rights

Thehelpfulbot will need admin rights to perform a delete of empty file pages. I'm happy to grant that, provided the script passes a script nomination. Does that seem reasonable? The one annoying thing will be the WMF policy of not allowing local bureaucrats to remove admin rights, after the bot run is finished. --Peter ^Talk 21:01, 19 December 2012 (UTC)

Thanks for posting this Peter, the first list of files to delete will be User:Thehelpfulone/Files to delete, and if the community is worried about removing admin rights, I'm happy to run this with a bot flag (that you can remove) from my own user account, User:Thehelpfulone or request a self-removal of the admin flag from Stewards on Meta-Wiki after this admin task is completed. Also the script I plan to be using is the open source Pywikipediabot, a well known and trusted set of scripts that is used across a large majority of Wikimedia Foundation wiki sets, the specific script will be delete.py, of which the latest source can be seen at <https://svn.wikimedia.org/viewvc/pywikipedia/trunk/pywikipedia/delete.py?view=markup>. Thehelpfulone 21:10, 19 December 2012 (UTC)

• Support. I'm OK with giving the bot admin rights, I'd just ask that you be even more careful than normal when deleting pages so that we don't have to undelete a few hundred files. The bot edits done thus far have all been extraordinarily useful, so thanks again for your work. -- Ryan • (talk) • 21:19, 19 December 2012 (UTC)
• Support per Ryan's rationale. And many thanks from my side for your continuous help to Wikivoyage. --Alexander (talk) 21:28, 19 December 2012 (UTC)
• Support. I initially made the request and I trust Thehelpfulone to do it well. This, that and the other (talk) 01:39, 20 December 2012 (UTC)
• Support. Also, some projects (such as these) allow bureaucrats to remove admin rights; we could request that ability here too. –sumone10154^(talk) 02:54, 20 December 2012 (UTC)

  Not to go too far off on a tangent, but of whom do we make such a request? Who decides whether we get the ability? --Peter ^Talk 06:55, 20 December 2012 (UTC)

  In terms of making this request, once community consensus has shown that this is wanted (for example the French Wikipedia had a support/oppose vote, then you can create a bug in Bugzilla using the Site requests component of the Wikimedia product. Given that previously bureaucrats had the "edit all user rights" flag, I'd presume you would like these similar rights to allow bureaucrats to add/remove adminship and bureaucratship - you wouldn't be able to get the "edit all user rights" as this would allow the addition of higher-level permissions such as oversight and CheckUser, which have more strict requirements. Thehelpfulone 12:49, 20 December 2012 (UTC)

  CheckUser and Oversight have to be done by stewards as there is an identification requirement from the WMF. --Rschen7754 20:01, 28 December 2012 (UTC)

  Indeed, sorry for not explaining fully, but because CheckUsers and Oversighters get access to non-public information, they are required to identify (send their real-life ID in) to the Wikimedia Foundation to confirm that they are over 18, as required by the Foundation's Access to nonpublic data policy. Thehelpfulone 20:38, 28 December 2012 (UTC)

I think there is consensus; are you happy to grant the right, Peter? Once the job is done, a post can be made at m:SRP to desysop the bot. This, that and the other (talk) 08:32, 27 December 2012 (UTC)

Thanks for granting the right Peter, this should now be done for all the pages at User:Thehelpfulone/Files to delete and User:Thehelpfulone/Files to delete 2. There is one image left that has the imagerepository="" result on the API, this is File:City.jpg but this is okay because it's protected placeholder on Commons. Thehelpfulone 15:19, 28 December 2012 (UTC)

Read-only programs section needs updating

The "Read-only programs" section needs updating as the terms of use have been replaced and the link to Project:terms of use#spiders points to a non-existent section. Nurg (talk) 09:06, 27 January 2013 (UTC)

I've changed this to point to m:Data request limitations. K7L (talk) 22:02, 2 February 2013 (UTC)

Allowing global bots

See m:Global_bots#Global_bots:

Global bots are given access on all wikis that allow global bots (this must be explicitly permitted by local policy, see list). To qualify for global bot access, the following requirements must be met:

• the bot must only maintain interlanguage links or fix double-redirects;
• the bot must already be active on several wikis, with long-term contributions to back up its trustworthiness.

Hoo User Page Bot is an example of such a bot - it is apparently a global bot that users can use to sync their user page across projects. I see no harm in allowing these sorts of bots to run here without a separate nomination, and would propose that English Wikivoyage be added to the m:Bot_policy/Implementation#Where_it_is_policy list - note that Russian Wikivoyage is already on the list. Here is the specific text the Meta asks that we use to approve this request:

To facilitate steward granting of bot access, I suggest implementing the standard bot policy on this wiki. In particular, this policy allows stewards to automatically flag known interlanguage linking bots (if this page says that is acceptable), which form the vast majority of such requests. The policy also enables global bots on this wiki (if this page says that is acceptable), which are trusted bots that will be given bot access on every wiki that allows global bots.

This policy makes bot access requesting much easier for local users, operators, and stewards. To implement it we only need to create a redirect to this page from Project:Script policy, and add a line at the top noting that it is used here. Please read the text at m:Bot policy before commenting. If you object, please say so; I hope to implement in two weeks if there is no objection, since it is particularly written to streamline bot requests on wikis with little or no community interested in bot access requests.

-- Ryan • (talk) • 17:53, 2 February 2013 (UTC)

• I don't see a problem with this - from my reading, it leaves the local process in place for local bots, but allows global bots to run without additional overhead. --Rschen7754 19:56, 2 February 2013 (UTC)

If a bot has gained global bot status, I can't think of a reason why we wouldn't grant it the same status here. --Peter ^Talk 20:13, 2 February 2013 (UTC)

The reasoning behind the global bot status is to allow the operators to run without having to apply for local status, as there are over 700 WMF wikis. This reduces the overhead of running a such bot. --Rschen7754 20:36, 2 February 2013 (UTC)

This thread has been active for over two weeks now. Any further comment before I make the request on meta to allow global bots to run here? -- Ryan • (talk) • 02:27, 20 February 2013 (UTC)

I've taken another look through and it seems that this actually only covers double-redirect or interwiki bots, not the sort of messaging bots that were the cause of concern. That being said, interwiki/double-redirect bots should be approved automatically so I support nonetheless. --Rschen7754 04:19, 20 February 2013 (UTC)

• Support. Pashley (talk) 04:21, 20 February 2013 (UTC)

I've submitted the request to add us to the global bot list: m:User_talk:Pathoschild#Request:_Global_Bots_for_English_Wikivoyage. -- Ryan • (talk) • 05:52, 22 February 2013 (UTC)

The request has been fulfilled - global bots will now be flagged to run on Wikivoyage without a separate nomination and any action required from bureaucrats here. -- Ryan • (talk) • 02:47, 23 February 2013 (UTC)