YouTube, bless them.

[jackandahat]

They're trying. They're really trying. But after yet another person went on about how autocaptioning was "close enough", I grabbed the last video I'd had open and took a look at just how close the autocaptioning was to what I could hear. (Disclaimer - I am hard of hearing, but I lipread pretty well, so I think I know what they were saying.)

15 errors in 25 seconds of video.

Several of these totally changed the plot, some were just damn silly.

Expanded on here, now with bonus Cosby Show!

Comments

[teafeather]

I was really excited about auto-captions when they first came out, as I have auditory processing problems. Unfortunately most of the captions are so bad that they leave me feeling even more confused about what's being said and interfere with the bits I can hear clearly.

[jackandahat]

They really just don't make sense. And the fact that the same phrase - "donut shop" was translated as both "amos don't" and "don't shout". It's not even consistently bad!

[codeman38]

Yeah, same here! Pretty much every video I see people link that only has auto-captions is practically incomprehensible, so much so that my dodgy auditory processing can make more sense of it.

Personally, I'm amazed more people haven't taken advantage of the auto-timing feature, where YouTube automatically adds timestamps to a straight transcript. That works surprisingly well, in fact-- because hearing already transcribed words is a much less intractable problem than making sense of audio with no hints at all.

[sophie]

I know I've definitely used it on one of my videos so far, and yes, it is pretty good :)

[maewyn]

I haven't done anything with autocaptioning, but I work as a closed captioner, and the number of times I have to back up the video and re-listen to figure out WTF people are saying... I am not surprised there are errors in the autocaptioning. I'd also guess that the error ratio would go up depending on how many speakers there are and whether they're using a specialized vocabulary (like on a home renovation show).

[jackandahat]

Except that's sort of my point - it's not specialised vocab. At all. It confuses things like "road" and "world", "stripped" and "straight", "donut shop" and "don't shout". I wouldn't call any of those particularly technical.

I understand that it's difficult - my issue is that people keep pointing to this and saying "But YouTube captions things!" if by captioning you mean 'putting words on it', sure. If you mean words that have any connection to what's being said...

[codeman38]

A few times, I've actually quoted the auto-captioning back at people just to give them an idea of how incomprehensible it is. It almost always comes as a surprise to them. (I mean, seriously. Do they not actually bother to try to watch things with captioning?)

[jackandahat]

No, no they don't - because they don't need to. So clearly, they know better than us about accesibility.

Hell - last time I quoted captions, someone said that if I could hear the audio was wrong then I obviously don't need captions. Because... clearly, the ability to hear one obviously wrong line while concentrating means I can't follow a full movie with ease.

[codeman38]

ARGH YES YES YES.

If what I hear is "mumble mumble donut shop mumble" and the captions include "don't shout" where I heard "donut shop", that obviously means I could make out the rest of the dialogue, right? >_<

[jackandahat]

Or if something is just so amazingly wrong that even if I can't work out what the speaker is saying, it's obvious it isn't correct.

One example that just came up in my journal - what do you think this sentence actually was? "are you going to run the business have really the Atlantic for that"

[codeman38]

I had no idea, but I would never have gotten the correct transcript that you gave in your journal out of that!

[jackandahat]

Yep. Now granted - the man saying that has what a lot of English and American people tell me is a heavy accent. (He sounds like half my uncles, so I'm biased.) But talk about mangled!

[maewyn]

Oh, I totally get you and I'm not defending it. I meant that specialized vocab would be even worse.

With the sheer number of videos on YouTube, it'd cost buckets of money to hire a real person to caption them all. Somebody to *correct* all those auto errors, however, would do some real good.

[jackandahat]

Ah, I get you. Sorry, I'm a bit defensive because it seems like every time this is brought up, someone goes "But you can't expect it to work!" which... isn't the point. We know it doesn't work, our problem is other people who insist it does!

What would be great is if you could user-correct. Sadly, this will never happen because half the internet is made up of the kind of knuckledraggers who would use this as a chance to put swear words on kiddie programs and such.

[maewyn]

I am a little horrified that anyone could think you'd get accurate captioning just from software! Have these people spent any time at all with voice recognition software and how you have to speak just right, and train the software to your voice, and how the speaker(s) in your average YouTube video are doing neither of these things?

It could be awesome if users could report captioning errors, or even just flag a video in general for review of the captions, and then have someone go in to fix the errors. (I can dream, right?)

[jackandahat]

No, they haven't. And it follows a long tradition of sweeping aside disabled people by going "Well, we did something for you, what do you mean it's not enough/not what you asked for/doesn't work???"

It would be great - though as things stand every video would be flagged, so it doesn't really work.

[kyrielle]

What would be great would be if someone watching a video and spotting captioning issues could provide an alternate transcript that would be stored for review by someone (so people couldn't, as mentioned above, put swear words on kiddie programs), and released once reviewed.

[codeman38]

The obvious solution to that would be moderation, so that the video producer could outsource them but would have to give the final OK for them to show up.

But of course, Google wouldn't bother to implement something like that, now, would they?

[treeowl]

They would need a lot more infrastructure than you're suggesting. How would the moderators be chosen? There would need to be an awful lot of moderators. How would they prevent the knuckle-draggers from overwhelming the moderators with a flood of crud?

[codeman38]

Ah, good point. I was thinking it could be handled just like comments, with the video editor doing all the moderation... but then I consider just how frustrating it is to have to delete horrible comments. -_-;;

[treeowl]

It wouldn't be bad to offer that as an option to the video uploader, I suppose, but I'm not suer it's the best way to handle it.

[codeman38]

It's also greatly affected by how much background noise there is. The voice-recognition system is trained for complete silence, and so of course it's completely thrown off if there's even the slightest bit of music in the background.

[jackandahat]

...OK, this is spooky, I think I was just reading you in another window. I dug up Consumerist's post about Up and captions to show a friend, and I was scrolling through comments. Are you Codeman38 there too?

[codeman38]

Yep! C'est moi. In fact, I'm the one who reported the missing captions to them.

[jackandahat]

And so I see now I'm paying more attention! It did seem like an unusual enough name that there probably wouldn't be two of you. It was a "Woah!" moment when you popped up in my inbox.

[nafs]

Autocaptioning is still horrible on TV (seriously, the captioning on The Weather Network is so bad I sometimes suspect it's an actual person who is bored and figures no one is reading and so types nonsense to entertain themselves) , I don't really get how Youtube thought it was going to be better on the internet.

[jackandahat]

That's interesting - I didn't realise they used autocaptioning there.

[nafs]

I don't know if they do in the States but on the Canadian it's basically the only explanation I can think of.

[jackandahat]

I'm in England - I haven't watched TV in years, but back in the 90s we had reasonable captioning - wasn't perfect, but I remember it being followable. So it surprised me.

[mercredigirl]

I just ran into this, because my brain has difficulty focusing on unsubtitled video, and I had to look at videos for work... I am appalled by the auto-captioning. I mean, if it doesn't work, why unveil it?

[jackandahat]

So they can look good? No, OK, I don't think it was malice. I think it's just... they didn't think about just how unuseable it would be.

[mercredigirl]

Maybe they didn't think people would actually need to use it. >

[jackandahat]

Pretty much. Two clashing stereotypes at work, I think - one, that only people's grannies need captions, and two, that there are no old people on the internet. Therefore if only old people need captions, and there are no old people on the internet, no-one on the internet needs captions.

We know that's bullshit, but considering the response most of us have had when asking places for captions, or daring to have hearing loss in public while <60...

[mercredigirl]

I know transcribing takes spoons, so I don't want to ask transcripts of everyone, but I do wish that - for media corporation products or whatever, at least - transcripts could be made available. :(