Commons:Village pump/Technical

From Wikimedia Commons, the free media repository
Jump to navigation Jump to search

Shortcuts: COM:VP/T • COM:VPT

Welcome to the Village pump technical section
Technical discussion
Village pump/Technical
 Bug reports
 Code review
Tools
 Tools/Directory
 Idea Lab



This page is used for technical questions relating to the tools, gadgets, or other technical issues about Commons; it is distinguished from the main Village pump, which handles community-wide discussion of all kinds. The page may also be used to advertise significant discussions taking place elsewhere, such as on the talk page of a Commons policy. Recent sections with no replies for 30 days and sections tagged with {{Section resolved|1=--~~~~}} may be archived; for old discussions, see the archives; recent archives: /Archive/2025/01 /Archive/2025/02.

Please note
 
SpBot archives all sections tagged with {{Section resolved|1=~~~~}} after 1 day and sections whose most recent comment is older than 30 days.

Template:Q+/doc on Wikispecies

[edit]

I imported {{Q+}} and Template:Q+/doc from this project into Wikispecies, but the latter is behaving very oddly, and the former not working as expected, and I cannot figure out why. Wikispecies is a small project with few technical editors; can someone assist there, please? Andy Mabbett (Pigsonthewing); Talk to Andy; Andy's edits 16:07, 19 December 2024 (UTC)[reply]

Can anyone assist, please? Andy Mabbett (Pigsonthewing); Talk to Andy; Andy's edits 11:18, 19 January 2025 (UTC)[reply]

Open files uploaded by a user all in new tabs

[edit]

How to open x recent files uploaded by a given user in new tabs at once? It takes very long to click all of them. Alternatively, a way to replace a category based on the file title would be great as well but it probably doesn't work as well. Here's what I intend to do:

going through the recent uploads of Florin Talasman to replace category "Agricultural products" with whatever product the map is about which varies per file (e.g. kiwis).

For example, maybe it's possible to output files uploaded by a user with Wikimedia Commons Query Service in a column with the URLs, then select all those URLs (maybe copy pasting it somewhere) and drag them into the Firefox tabs to open them all in new tabs. That didn't work because I could only query for files created by the user but not files uploaded by that user (see also Make it possible to search by page author /contributor/ uploader).

I think this is generally useful, not just for this particular example case. Prototyperspective (talk) 22:58, 28 December 2024 (UTC)[reply]

If you have all the links (e.g. from Special:Log/upload), you can use an extension like Linkclump or [1] to open them all in new tabs. —‍Mdaniels5757 (talk • contribs) 23:11, 28 December 2024 (UTC)[reply]
I can just drag and drop the URLs once I have them all marked to open them all in new tabs in Firefox. The problem is in getting the plain URLs. Prototyperspective (talk) 23:17, 28 December 2024 (UTC)[reply]
for(const d of document.querySelectorAll('.TablePager_col_img_name a:nth-child(1)')) console.log(d.href);
output https://litter.catbox.moe/2we6sw.txt 999real (talk) 15:19, 31 December 2024 (UTC)[reply]
Is this javascript to be used in the Web console of the recent uploads page (Special:ListFiles) or where? Prototyperspective (talk) 16:49, 31 December 2024 (UTC)[reply]
Yes in console on Special:ListFiles pages
This one you can save as a bookmark and it will write to clipboard directly
javascript:(function() { navigator.clipboard.writeText(Array.from(document.querySelectorAll('.TablePager_col_img_name a:nth-child(1)')).map(link => link.href).join('\n')) })(); 999real (talk) 22:42, 31 December 2024 (UTC)[reply]
Thank you! That's very helpful and I guess solves this issue. It should probably be in some relevant Help page so people looking for this functionality can find it. I think one can't filter files to only copy files that are videos of a specified resolution that way (which would be a workaround to solve phab:T377606) so a way to search files uploaded by a given user would still be useful. Prototyperspective (talk) 22:47, 31 December 2024 (UTC)[reply]
I tried write one but Special:ListFiles does not show the resolution directly, this is very slow because it has to load video metadata to fetch the width and height
(function() {
	function getVideoDimensions(videoUrl) {
		return new Promise((resolve, reject) => {
			const videoElement = document.createElement('video');
			videoElement.src = videoUrl;
			videoElement.addEventListener('loadedmetadata', () => {
				const width = videoElement.videoWidth;
				const height = videoElement.videoHeight;
				resolve({ width, height });
			});
			videoElement.addEventListener('error', (e) => {
				reject(`Error loading video: ${e.message}`);
			});
			videoElement.load();
		});
	}
	const uploads = document.querySelectorAll('.TablePager_col_img_name a:nth-child(2)');
	const toCopy = [];
	const promises = [];
	for (const d of uploads) {
		const url = d.href;
		if (url.endsWith('.mpg') || url.endsWith('.mpeg') || url.endsWith('.ogv') || url.endsWith('.webm')) {
			const promise = getVideoDimensions(url).then(dimensions => {
				const minimumWidth = 1000;
				const minimumHeight = 720;
				if ((dimensions.width < minimumWidth) || (dimensions.height < minimumHeight)) {
					toCopy.push(url);
				}
			}).catch(error => {	
				console.error(error);
			});
			promises.push(promise);
		}
	}
	Promise.all(promises).then(() => {
		if (toCopy.length === 0) {
			alert('Nothing to copy');
		} else {
			navigator.clipboard.writeText(toCopy.join('\n')).then(() => {
				alert('Copied ' + toCopy.length + ' items');
			}).catch(err => {
				console.error('Failed to copy: ', err);
			});
		}
	});
})();

Then I found VisualFileChange interface actually shows video dimensions, this is much faster

(function() {
	const toCopy = [];
	const titles = document.getElementsByClassName('jFileTitle');
	const dimensions = document.getElementsByClassName('jFileSize');
	
	const minimumWidth = 1000;
	const minimumHeight = 1000; 
	
	for (let i = 0; i < titles.length; i++) {
		const url = titles[i].href;
		if (url.endsWith('.mpg') || url.endsWith('.mpeg') || url.endsWith('.ogv') || url.endsWith('.webm')) {
			const ds = dimensions[i].textContent.replace(/^.*KiB/, '').replace('px', '');
			const width = ds.replace(/^.* x /, '');
			const height = ds.replace(/^ x .*/, '');
			if((minimumHeight > parseInt(height.replaceAll(' ', ''))) || (minimumWidth > parseInt(width.replaceAll(' ', '')))) toCopy.push(url);
		}
	}
	if (toCopy.length === 0) {
		alert('No videos with width below ' + minimumWidth + ' or height below ' + minimumHeight + ' found on this page');
	} else {
		navigator.clipboard.writeText(toCopy.join('\n')).then(() => {
			alert('Copied ' + toCopy.length + ' items');
		}).catch(err => {
			console.error('Failed to copy: ', err);
		});
	}
})();

999real (talk) 00:09, 1 January 2025 (UTC)[reply]

@Prototyperspective for this specific case, since what you want can be extracted from the title, you could use com:vfc to batch replace "Category:Agricultural products" with "Category:{{subst:#invoke:String|sub|s={{subst:PAGENAME}}|j=-22}}s", and then fix any possible errors. RoyZuo (talk) 08:30, 1 January 2025 (UTC)[reply]
Thanks so much! I'll spend some time trying to use this to fix videos at a later point. I linked it here. I haven't uploaded so much that this wouldn't solve the problem for me but it's not ideal to have to go through each page manually; maybe at some point this could be a Web UI tool on toolforge where one enters a username and then something to search for where this js is then used to find all the results. The earlier script / the txt also work as intended; already added some categories to filepages opened using that and will get to completing that later. Prototyperspective (talk) 17:28, 16 January 2025 (UTC)[reply]

Twinkle

[edit]

Hello, I’ve noticed that some files have been created solely for advertising purposes. When I used Twinkle to request speedy deletion under criterion G10, the tool displayed the message: "Tagging page: The edit was disallowed by the edit filter: 'Protect page section headings'." I’m unsure what this means, and in this case, the tool wasn’t able to apply the speedy deletion template. I would greatly appreciate it if you could help clarify this for me. Thank you so much, and have a nice day! P. ĐĂNG (talk) 07:43, 3 January 2025 (UTC)[reply]

You tried to replace the hole page content with the speedy deletion template. All deletion templates have to be added at the first line of the page content and the content must not be removed. GPSLeo (talk) 10:15, 3 January 2025 (UTC)[reply]
Just because something was upladed for advertising purposes does not mean that it is not usable. --MGA73 (talk) 18:19, 27 January 2025 (UTC)[reply]

Manual license template in UploadWizard hangs

[edit]

When I enter a manual license template in the UploadWizard, and want to move on, the Wizard hangs on validating the license template. Looks like a bug --PantheraLeo1359531 😺 (talk) 09:17, 10 January 2025 (UTC)[reply]

These messages appear in console:
Uncaught TypeError: status.getErrors is not a function
jQuery.Deferred exception: this.getUsedTemplates is not a function TypeError: this.getUsedTemplates is not a function
jQuery.Deferred exception: status.getErrors is not a function TypeError: status.getErrors is not a function
Google Chrome 126.0.6478.126 Radmir Far (talk) 15:50, 10 January 2025 (UTC)[reply]
@Ebrahim: is this related to your recent changes? —‍Mdaniels5757 (talk • contribs) 19:58, 10 January 2025 (UTC)[reply]
UploadHelper.js is unrelated to the UploadWizard, sorry. —‍Mdaniels5757 (talk • contribs) 20:04, 10 January 2025 (UTC)[reply]
Also experiencing this PascalHD (talk) 20:17, 10 January 2025 (UTC)[reply]
This is on Phabricator now. —‍Mdaniels5757 (talk • contribs) 22:22, 11 January 2025 (UTC)[reply]

Bugged cache

[edit]

Please purge the cache of File:Color Multicircle.svg. Unfortunatley 4 "null edits", two "action=purge":s, two reverts and 4 moves were not sufficient. The circle below "File:Color Multicircle.svg has 15 code variations:" is still showed with wrong colour. Taylor 49 (talk) 21:50, 11 January 2025 (UTC)[reply]

The problem is not visible anymore. Still the increasing phenomenon of various fact-resistant and purge-resistant caches on wikis should be addressed. Taylor 49 (talk) 13:10, 12 January 2025 (UTC)[reply]
It is likely that this was browser cache (client cache, instead of server cache). —TheDJ (talkcontribs) 20:26, 14 January 2025 (UTC)[reply]
Also, i don't know who came up with the idea of using arbitrary language values in order to vary an SVG, but... please realize that this isn't supported functionality and ma break at any time. —TheDJ (talkcontribs) 20:31, 14 January 2025 (UTC)[reply]
Several people have come up with the idea. IIRC, there's even a Phab ticket. Such files should be deprecated. Glrx (talk) 22:37, 14 January 2025 (UTC)[reply]

Problems with file upload when inserting templates from different sites

[edit]

Good day! My files stopped loading instantly when I inserted a template from a specific site (for example, kremlin.ru). Can you explain the cause of the problem? MasterRus21thCentury (talk) 10:50, 12 January 2025 (UTC)[reply]

I don't understand the request. No WMF wiki can invoke templates from "kremlin.ru" if such happened to exist. Taylor 49 (talk) 13:10, 12 January 2025 (UTC)[reply]
Seems to be a problem with the UploadWizard, see phab:T383415. Radmir Far (talk) 13:55, 12 January 2025 (UTC)[reply]

Tech News: 2025-03

[edit]

MediaWiki message delivery 01:38, 14 January 2025 (UTC)[reply]

[edit]

The image File:Cassowary_Diversity.jpg doesn't display at most resolutions. The page looks okay, but if I click on "Open in Media Viewer" I see an error message: "Sorry, the file cannot be displayed: There seems to be a technical issue. You can retry if it persists. Error: could not load image from https://upload.wikimedia.org/wikipedia/commons/thumb/4/4b/Cassowary_Diversity.jpg/1920px-Cassowary_Diversity.jpg". If I try to go to that URL, or go to any of the other resolutions on the File: page, I get "Unauthorized: This server could not verify that you are authorized to access the document you requested."

This breakage can also be seen at https://en.wikipedia.org/wiki/Cassowary on Wikipedia. Danbloch (talk) 03:18, 19 January 2025 (UTC)[reply]

This is also a problem at File:Status_iucn3.1_CR.svg, and File:NuclearReaction.svg, as reported at the enwiki Teahouse by @MtBotany: . It looked like maybe only SVGs were impacted but it appears it's also other filetypes. Berchanhimez (talk) 03:43, 19 January 2025 (UTC)[reply]
It is phab:T384128, the Swift container holding 4b thumbs is corrupted. Dylsss (talk) 04:34, 19 January 2025 (UTC)[reply]
Thanks. I noticed you added the tracked onto the Teahouse right before I came here with it. I swear I wish those tracked boxes had a little bit more different background, because I totally missed it. Hopefully the technical gremlins get put back to rest soon :) Berchanhimez (talk) 04:41, 19 January 2025 (UTC)[reply]

Broken image, thumbnails not available

[edit]

File:E-3. Mormon Monument (I-80 Rest Stop, Lyman, WY) on the Mormon Pioneer National Historic Trail (2007) (46fef7c9-2a82-42db-be2d-389b214ca161).jpg doesn't display any more. --Bean49 (talk) 13:33, 22 January 2025 (UTC)[reply]

This is actually a different unrelated issue: phab:T381594. Dylsss (talk) 01:21, 23 January 2025 (UTC)[reply]

Tech News: 2025-04

[edit]

MediaWiki message delivery 01:33, 21 January 2025 (UTC)[reply]

UploadWizard mistaking license on Flickr uploads

[edit]

So earlier I used my alt account to upload this file (https://www.flickr.com/photos/presidenciaecuador/54268607515/) with UploadWizard, and when I finished and clicked the file page open the license it showed was PD-US instead of the normal CC0 license on Flickr. Special:Diff/985808004. Did anyone else encounter this as well? Not sure whether it is related to the earlier cases of UploadWizard messing up licenses above. 沪A 05683DS5A-0043 07:34, 21 January 2025 (UTC)[reply]

That file is licensed on Flickr as PDM, not CC0. I don't think Upload Wizard was ever updated to reflect that licensing. The correct license tag is {{PD-author-FlickrPDM}}, which I add manually whenever I have time to revisit such uploads. RadioKAOS / Talk to me, Billy / Transmissions 07:45, 21 January 2025 (UTC)[reply]
Oops, read wrongly (facepalm). I’m pretty sure UploadWizard does have the PDM though-or at least back in 2023 when I uploaded this file. 沪A 05683DS5A-0043 09:58, 21 January 2025 (UTC)[reply]
So is this issue solved? Prototyperspective (talk) 15:33, 26 January 2025 (UTC)[reply]
Not quite. See Special:Diff/990789717 which I newly uploaded. S5A-0043🚎(Leave a message here) 09:19, 31 January 2025 (UTC)[reply]

Tech News: 2025-05

[edit]

MediaWiki message delivery 22:11, 27 January 2025 (UTC)[reply]

Regions of Belarus

[edit]

Hi! Could you help with the module for categories by year for regions of Belarus, Kazakhstan, Uzbekistan, Kyrgyzstan and Tajikistan on {{Region year}}? This is the equivalent of the template {{Oblast year}} for Russia and Ukraine. MasterRus21thCentury (talk) 20:23, 29 January 2025 (UTC)[reply]

Category for City by year

[edit]

How do I go about creating or editing a new category for a city/town by year? For example, when viewing Category:Oshawa by year, select 2024 for example and hit 'Edit' all you can see is "CanadaYear|Oshawa|202|4" which automatically adds the bar at the top to filter by year and categories. Is there a template somewhere? PascalHD (talk) 20:46, 1 February 2025 (UTC)[reply]

Creator Possible error

[edit]

Unless I'm missing something, it seems that Template:Creator possible isn't properly putting things in Category:Creator templates to be created by a bot when it should be. ToxicPea (talk) 04:42, 2 February 2025 (UTC)[reply]

"Defective" files

[edit]

There's an issue with a lot of video files on Commons where, if a video file does not have metadata associated with it, it shows as "0.0s" in length. This issue's intensity can vary from the video file being watchable but not entirely technologically readable across WMF projects, to a video file not even being able to skip forward, but only play in one go.

I think this is probably some kind of bug in the Wikimedia video software. If a file was not given metadata, you can fix this by using the following FFMPEG command in your terminal. For example:

ffmpeg -i The_Crisis_\(1916\).webm -metadata title="The Crisis" -metadata year="1916" -metadata artist="Colin Campbell" -metadata genre="Drama, Historical" -metadata description="A silent historical drama film directed by Colin Campbell, based on the novel by Winston Churchill." -metadata comment="Silent film classic from 1916." -c copy The_Crisis_\(1916\)_metadata.webm

I feel like any bogus data being thrown in there should be fine. I did this for File:The Crisis (1916).webm and the issue was fixed immediately.

This issue, with "defective" files (that are in my opinion not fully defective but the software treats them as if defective) can lead to video files being dangerously deleted, such as File:Old Ironsides (1926).webm once was, before I saved it with a higher quality version. This sets a precedent for these "0.0s defective files"—of films that are often quite rare finds and may no longer exist anywhere but Commons!—to be simply speedied without notice.

I think the solution to this problem is to add to the list of Commons bots a "fix defective file bot", that simply adds some metadata to the video file (possibly based on what's in the Commons description or just random data), so that the number of files with this problem doesn't keep building up. Or, we could fix the WMF video technology, to where the backend automatically populates the metadata of a metadata-less file.

At the very least, is there a category that keeps up with files listed as "0.0s"?

Nevertheless, this is a recurring issue, and I want to see if there's some way to fix this problem quickly using a systemic method (bots, site code, etc.)? Pinging @Racconish and Yann: who have noticed video files with this kind of defectiveness before as well. SnowyCinema (talk) 16:28, 2 February 2025 (UTC)[reply]

The solution is either a solid investment of hardware, software and staff in being able to handle video decoding in a more efficient way or stop uploading video all together. This is such a recurring topic and we're just wasting resources at the moment. Sjoerd de Bruin (talk) 17:51, 2 February 2025 (UTC)[reply]

‘-i---i-’ prefix in filenames

[edit]

I happened to see the ‘-i---i-’ prefix in filenames in different contexts, so I had a look. There seem to be a lot of files where their names all start with ‘-i---i-’ and they were all uploaded using Flickr2Commons, but they are otherwise unrelated. What’s going on? Brianjd (talk) 07:11, 3 February 2025 (UTC)[reply]

Probably some bad import from the Flickr. I ocasionally rename some of these files, but it's a Sisyphean work. Some bot action would be really appreciated, e.g. renaming the files with the Description field as a filename. — Draceane talkcontrib. 09:07, 3 February 2025 (UTC)[reply]
Hello @Brianjd
If you come across such files and have a better description of what the image represents, please suggest a rename. Greetings זיו「Ziv」For love letters and other notes 04:06, 4 February 2025 (UTC)[reply]
@Ziv: I will (for files I care about enough), but that’s sort of missing the point. The question is not what to do with these filenames (we all agree they should be changed), but where they come from in the first place. Brianjd (talk) 06:39, 4 February 2025 (UTC)[reply]
I'm not sure where the precise string "-i---i-" comes from, but it usually seems to show up on untitled Flickr images, e.g. File:-i---i- (50282028261).jpg was imported from this untitled photo. It'd be nice if Flickr2Commons could be a little pushier about requiring users to enter titles for these imports. Omphalographer (talk) 00:08, 7 February 2025 (UTC)[reply]
@Omphalographer: Yes, that does seem to be it. Sometimes it is even worse: -i---i- (32552103431).jpg has no title and also has no description. Brianjd (talk) 07:44, 7 February 2025 (UTC)[reply]

Tech News: 2025-06

[edit]

MediaWiki message delivery 00:05, 4 February 2025 (UTC)[reply]

Unrealistic minimal length requirement (minimal number of characters) for file captions in East Asian languages

[edit]

if you know the cjk script, you will know that many words are just 2 characters long. spider = 蜘蛛, North America = 北米... many are even only 1 char. oranges = 橙, mountain = 山...

for all these examples, the english words can become files' english captions (string length > 5), but the kanjis cant be accepted because length < 5.

it's been many years since i filed this bug and it's still bugging me even though i rarely write cjk captions. cant imagine how annoying this is for users who write in those languages. when will this alphabet-centric problem be solved? RoyZuo (talk) 20:22, 4 February 2025 (UTC)[reply]

Low-resolution old images from the Library of Congress

[edit]

People periodically upload the low-resolution files available from the Library of Congress (or in some cases the LoC makes available better files). I've tried (well to be honest I've used ChatGPT, as I know no SQL) on-and-off to formulate a Quarry query for this for several months now, to no great success.
My ideal search would be something like what follows:

  • Resolution under 2MP (not picky here)
  • Permission or source template has a link to the Library of Congress or uses one of the LoC templates with ID.
  • The file is in use on a Wikimedia project

I've been experiencing timeouts of my queries (among other issues(, so my parameters are either too complicated or botched by ChatGPT. Has anyone technical expertise enough to take a look? JayCubby (talk) 18:46, 5 February 2025 (UTC)[reply]

I can try to do a better query. Can you give some example image for the starting point? --Zache (talk) 19:09, 5 February 2025 (UTC)[reply]
The most straightforward-seeming images are ones like File:Albert Dillon Sturtevant in 1918.jpg, File:Col Charles Lindbergh.jpg, or File:CharlesRMabey.jpg which use {{LOC-image}}
Examples of 'bad' source listings are files like File:Two Winnebago men by WH Illingworth.jpg (the listed source URL is this, but the proper file can be found by searching 'two winnebago men illingworth')
File:Dixie 5249224624 c423d81d41 o.jpg is somewhere between straightforward and weird, with the LoC description copy-pasted.
Hope this helps,
JayCubby (talk) 19:24, 5 February 2025 (UTC)[reply]
Here is one only with images that use template:loc-image, limited to max 1414px width and height, which is just below 2MP. A list of Libary of Congress templates that have an ID parameter would be nice, it is not something I can query. Query took for me 28 seconds.
select page_title from page
join image on page_title = img_name and img_width < 1414 and img_height < 1414
join templatelinks on tl_from = page_id and tl_target_id = 1396
join globalimagelinks on gil_to = page_title
where page_namespace = 6
group by page_title limit 1000
Snævar (talk) 19:36, 5 February 2025 (UTC)[reply]
This one way to do the traightforward case (ie same than you did but with ID). It is not faster than your version, but seems to be solid without timeouts
https://quarry.wmcloud.org/query/90505
--Zache (talk) 20:39, 5 February 2025 (UTC)[reply]
Also if you need only first 1000 you can do it like this. Basically it does the filtering the results based on templatelinks and externallinks first and limit number of results to wanted number in subquery. Then in second part it will filter the results to images used in wikipages using using globalimagelinks.
- https://quarry.wmcloud.org/query/90518 Zache (talk) 10:43, 6 February 2025 (UTC)[reply]
Thanks a ton! I will run these through Glammorgan and deal with the top-used ones, unless someone can automate this. The LoC has an API, which might be of help to more technical folks. JayCubby (talk) 14:00, 7 February 2025 (UTC)[reply]
@JayCubby: In the case of File:Col Charles Lindbergh.jpg, you should upload the original and the cropped versions separately. Thanks, Yann (talk) 09:26, 7 February 2025 (UTC)[reply]
Done! JayCubby (talk) 14:01, 7 February 2025 (UTC)[reply]

disappearing thumbnails for pdf/djvu files

[edit]

from all my 17 pdf uploads there are only 4 left with thumbnails at the moment. some of thumbnails disappeared in minutes after uploading, some in weeks. i had vague hopes it will be fixed by itself in time, but this magic works one-way only so far.

the issue seems to be seriously impacting the general search in "other media" tab. for example: "koltsov" or "Кольцов" search leads to "Invalid search Could not normalize image parameters for Aleksey_Vasilyevich_Koltsov_Yego_zhizn_i_sochineniya_1914.pdf" message.

attempts to search for "moskovskij zhurnal", "Московский журнал", "teleskop", "телескоп" lead to the same 'invalid search' result

while trying to search for "брюсов" i found another user affected by the same issue. the search led to "Invalid search Could not normalize image parameters for Первобытный_Брюсов_календарь_(1875).pdf" message.

some of the flies uploaded by HareGovorittKrishna with disappeared thumbnails arent ruining the search - theyre just staying invisible for it:

but some are: check lomonosov/ломоносов search.

none of the old HareGovorittKrishna's uploads are affected - so the issue mightve been started since at least november 2024.

so now my only way to upload while not ruining anything is uploading files with nonsense names, but i have a real distaste for the idea. TheyStoleMyNick (talk) 17:03, 8 February 2025 (UTC)[reply]