lardbucket: getting complicated, embedded youtube videos

9/1/2012

Getting Complicated, Embedded YouTube Videos

Filed under: Hacks, School, Technology — Andy @ 3:29 pm

So, earlier today, Karl Fisch asked for a copy of an Olympics video from NBC’s website to use in his school. A number of the “standard” ways of getting the video didn’t seem to work, so I figured I’d pitch in. By the way, the first thing to try (if whatever addons you might have to download YouTube videos don’t work – although I don’t use any myself, I’m told they didn’t work in this case) is to click the YouTube button.

The YouTube button will usually open the video up into YouTube’s website, where most addons for downloading a video will work better, and there are plenty of other websites happy to help you.

In this case, clicking the button only got to the NBC Olympics home on YouTube. A quick search of the channel showed that the video I was looking for wasn’t on the NBC Olympics channel, and I’d have to find another way to download it.

So, this is where Chrome‘s Developer Tools come in handy. For what we want, you’ll want to be on the page that has the video in question (the NBC Olympics page), and then go to the Chrome menu (it’ll either look like a wrench/spanner, or three lines, depending on your version of Chrome), then Tools, and Developer Tools. That’ll pop open a bunch of stuff, but go to the top and click Network. Then press refresh on the NBC Olympics page. You should have a page in Developer Tools that looks something like this:

This window still doesn’t give you what you want. (You may have noticed the “videoplayback” loading there, but it’s unfortunately probably not what you’re looking for, unless you relish the thought of assembling a bunch of chunks of data together by hand, and then adding on the appropriate header.) Instead, scroll up until you find a “get_video_info” on the left column, and click on it. That’s YouTube’s player requesting information about a video. The first one you’ll come to as you scroll up should be something like this (although I’ve blacked out my cookies):

Unfortunately, the “el=adunit” shows that this is actually the advertisement video that showed before the actual video we were looking for. If you scroll up to the next “get_video_info,” you’ll get one that looks like this:

So, no “adunit” means that this is probably the right video. Now, we need the contents of that file. For whatever reason, Chrome’s Developer Tools doesn’t actually show it, so we need to download it. The easy way to do this is to copy the whole “Request URL,” and paste it back into Chrome’s address bar. You should get prompted to download it. Save it somewhere you can find it, and open it back up in a text editor. You’ll have a bunch of random-looking text, that starts roughly like:

account_playback_token=&ptk=nbcolympics2012&focEnabled=1&vid=GLEM9kOrmK4&mp vid=AATIqPo-c4fWYloT&rmktEnabled=1&allow_ratings=0&keywords=FORscene&track_ embed=1&uid=AsteRGSg5ssDRYhsdsg&view_count=18410&no_get_video_log=1&fmt_lis t=46%2F1920x1080%2F99%2F0%2F0%2C37%2F1920x1080%2F9%2F0%2F115%2C45%2F1280x72 0%2F99%2F0%2F0%2C22%2F1280x720%2F9%2F0%2F115%2C44%2F854x480%2F99%2F0%2F0%2C 35%2F854x480%2F9%2F0%2F115%2C43%2F640x360%2F99%2F0%2F0%2C34%2F640x360%2F9%2 F0%2F115%2C18%2F640x360%2F9%2F0%2F115%2C5%2F320x240%2F7%2F0%2F0%2C36%2F320x 240%2F99%2F0%2F0%2C17%2F176x144%2F99%2F0%2F0&title=Highlights%3A+Missy+Fran klin+

…and goes on and on. (I’ve added spaces so that it wraps correctly.) If so, you can close Chrome’s Developer Tools for now, you don’t need them any more.

So, the file you have open does in fact contain everything you need to get the video you want. Unfortunately, it’s still hard to read. Go to Stephen Ostermiller’s handy encoder/decoder, and copy the contents of your get_video_info file into there. You can safely click “Decode” next to “URL” twice (it’s toward the bottom-left of the screen). At this point, you’re getting closer. Now you have things that look like actual URLs for files you could download. Pick out the “right” one, and you’re all set.

Which one’s the right one? Well, it’s best to figure out what you want. If you look through the text, you’ll see things like “quality=hd1080&fallback_host=tc.v16.cache1.c.youtube.com&type=video/webm” or “quality=hd720&fallback_host=tc.v20.cache5.c.youtube.com&type=video/mp4”. The quality settings are just the sizes for the video that YouTube has available. The “type” is the type of video. WebM, although it’s a great format, is a bit less commonly supported than MP4, so if you’re not into playing with new things yet, you’ll want one with “type=video/mp4”. The quality is the size you’ll get, and the fallback_host doesn’t matter. Once you’ve found the quality and type you’re looking for, go back to just before the quality. Select the whole thing from just before the “quality” to the first “url=” before it. For me, that would look like:

http://o-o—preferred—iad09s13—v19—nonxt3.c.youtube.com/videoplaybac k?upn=WunvRIvlpg0&sparams=cp%2Cgcr%2Cid%2Cip%2Cipbits%2Citag%2Cratebypass%2 Csource%2Cupn%2Cexpire&fexp=909142%2C926239%2C900327%2C922444%2C920135%2C91 2671%2C924238%2C913354%2C912126&key=yt1&itag=22&ipbits=8&signature=3223A363 72C223434223A36382C223435223A363.3A37312C223438223A37322C223439223A37332C&m v=m&sver=3&mt=1346156781&ratebypass=yes&source=youtube&ms=au&gcr=us&expire= 1346615516&ip=4.2.2.2&cp=GRpdiBzdHlsZT0icGFkZGluZzowLjJjbTt0ZXh0L&id=32g18y n918ui15yn&newshard=yes

(I’ve changed a few details and added spaces, but the general layout is the same.) Note that if your URL ends in any of the “&quality=[…]” bits from before, you need to remove that. It should end with “newshard=yes” (at least in this case, although that may change later).

Finally, take that URL, plug it into whatever you’d normally use to download a video (including your web browser, if it will let you save files), and save it. You should now have a video file of the video you wanted. (If you chose an MP4 file, you’ll want to give it an .mp4 extension so that more players will accept it. WebM uses .webm, and FLV uses .flv.) If you’ve gotten that far, and are still having trouble viewing the video, VLC is a great video player that will play pretty much anything you can throw at it.

The things we go through for Fair Use.

Please note: This procedure works as I’m writing this. It may not work in a year, and it may not work in an hour. It works at the moment, and I’m not going to be able to update this post every time it changes. This is just an indication of one way to get videos that are hard to get, and shows some of the work that goes into doing it. If things change, some of the tools here may give you enough to go on that you can actually find them yourself in the future. They may not. Unfortunately, I probably won’t be in the position to create new versions of this when something changes. Sorry.

2 Comments »

  1. Hi,
    I am trying to built a website where user can download Youtube videos. I have really worked very hard to learn how to code in PHP and get videos from Youtube.

    I am getting all the information about the video with this function:
    parse_str(file_get_contents(‘https://www.youtube.com/get_video_info?video_id=’.$id),$info);

    But the problem is when I refresh the page sometimes I get the “quality” and “type” parameter’s values and sometime not.

    What it going on please help me.

    Comment by Kelly — 6/8/2015 @ 2:02 am

  2. Kelly: Unfortunately, I can’t really help there. You might take a look at the code from youtube-dl, which is generally considered to be one of the better tools for downloading YouTube videos. The video I’m discussing didn’t work with youtube-dl back when I wrote the post, but it probably does now: the youtube-dl developers definitely keep up with those changes much better than I can.

    Comment by Andy — 10/26/2015 @ 5:04 pm

RSS feed for comments on this post.

Leave a comment

XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

My Stuff
Blog Stuff
Categories
Archives