Weblog of Abdullah Rubiyath :: Alif

Exploring the Web: An apple a day

IMDB Details Grabber: Using PHP DOM XPath to extract Movie details

Tweet
Thu, 05/21/2009 - 15:20 -- alif

Recently, a friend of mine asked me if there's an IMDB Parser that allows to fetch information about any Movie on IMDB. Unfortunately, IMDB does not offer an API, so , after doing a bit googling I found nice IMDB Grabber (PHP IMDB Grabber), despite it being a good one, I decided develop my own class relying on XPath and Dom Traversing.


Brief Explanation of Methods:

The Class 'class.imdb.php' has 3 useful Methods (in addition other ones). They are:

get($url) Fetches the contents of a specific Movie on IMDB. Call this Method by passing the URL and this Method would return as associate array containing the title, image, url, Plot, Director, Release Date and Runtime.
useCSV($status) By Default, when get Method is invoked, values returned are in CSV format. If this Method is invoked with false value as parameter, the values returned from get($url) Method would be a list of array, For example: if a Movie has multiple Directors, by default, it when get($url) Method is invoked, the value of 'Director:' would be a CSV containing multiple Director Names. If useCSV(false) is invoked and then get($url) is called, the associative value of 'Director:' would be an array containing list of Directors. Please see Usage for more info.
showCast($t) The parameter is a boolean. If you wish to grab the cast (some people have requested it), then call this method with a 'true' parameter. Then, when you grab an imdb page, the returned value would also contain the cast in the movies with their thumb image, real name and cast name.
showRating($t) The parameter is a boolean. If you wish not to grab the rating, then call this method with a 'false' parameter.

Usage (Basic):

Using the class is very easy. Include the class file, initialize the class and then invoke get($url), where $url is the IMDB URL for movie.

include 'path/to/class/class.imdb.php';
$imdbObj   = new Imdb();

// movieInfo contains the details of Movies in associative array format
$movieInfo = $imdbObj->get('http://www.imdb.com/title/tt0167260/');

// OR simply enter the movie name :)
$movieInfo = $imdbObj->get('The Matrix');

Here's a vardump of movieInfo:

Array
(
    [title:] => The Lord of the Rings: The Return of the King (2003)
    [image:] => http://ia.media-imdb.com/images/M/MV5BMjE4MjA1NTAyMV5BMl5BanBnXkFtZTcwNzM1NDQyMQ@@._V1._SX94_SY140_.jpg
    [url:] => http://www.imdb.com/title/tt0167260/
    [Plot:] => The former Fellowship of the Ring prepare for the final battle for Middle Earth, while Frodo & Sam approach Mount Doom to destroy the One Ring.|
    [Director:] => Peter Jackson
    [Release Date:] => 17 December 2003 (USA)
    [Runtime:] => 201 min  | 251 min (extended edition)
)

That's it, simple enough!. By Default, title, image, url, Plot, Director, Release Date and Runtime is returned. All Keys of the associative array as suffixed with ':' (colon), I leave it upto developers to either strip it out, or just display it.

Usage (Customization and Advanced Usage):

By default, the class fetches the title, image, url, Plot, Director, Release Date and Runtime, Aspect Ratio, Writers, from IMDB.

If you wish to grab the cast in the movie, then call the following method:

$imdbObj->showCast(true);

CSV: By Default, all Values returned are in CSV Format, if you want to get value as list of array, for example for Writers, you way want to get the list of Writers as in Array instead of CSV. In that case do the following:


$imdbObj   = new Imdb();
$movieInfo = $imdbObj->showCast(true)
                                     ->useCSV(false)
                                     ->get('http://www.imdb.com/title/tt0167260/');

View Demo Download from Github

Attachment Size
test.imdb.php.txt 642 bytes
class.imdb.php.txt 5.87 KB
demo.php_.txt 2.13 KB

Comments

thanks :-) i'll try to use it

Submitted by Hossain (not verified) on

thanks :-)
i'll try to use it soon...

Error with PHP

Submitted by Anonymous (not verified) on

For some reason, I'm getting the following error:
Warning: DOMDocument::loadHTMLFile() [domdocument.loadhtmlfile]: htmlParseEntityRef: no name in http://www.imdb.com/title/tt0167260/, line: 642 in class.imdb.php on line 146

Over and over and over again, but the result does show at the end.

Thanks, I will look into it

Submitted by alif on

Thanks for trying it out. From the messages on your comment, it seems to be a warning, not an error. I will look into it. For the time being, if you wish the warning message not to be displayed, you can use error_reporting feature of php to not show the warning.

http://us2.php.net/manual/en/function.error-reporting.php

Doesn't seem to work !

Submitted by Susan (not verified) on

The code doesn't seem to work, I get error messages, it cannot find DomDocument ? Where is that declared..???

Can you elaborate more on the error?

Submitted by alif on

Can you please explain what error you are getting. DomDocument is the standard Class of PHP-5. Ensure that you are using PHP-5.1 atleast.

I think yours will be even

Submitted by cash loan (not verified) on

I think yours will be even better than these alternatives. Looks pretty straight forward.

cant get the cast

Submitted by Anonymous (not verified) on

Hi there

i have used your script. it works great. but whatever i do i cant retrieve the CAST of the movie from that page. Please let me know if its possible
cheers

Awesome!

Submitted by Johnson (not verified) on

This is really awesome. Really easy to use.

Thanks

nice writing

Submitted by payday loan canada (not verified) on

This piece of writing is excellent and I liked it a lot.

Requesting some help

Submitted by Ahmed (not verified) on

Thanks for making this. I have been looking for something like this to manage my persona movie collection using PHP / MySQL.

When I put your 2 files in my xampp/htdocs/moviecritc folder and run it through http://localhost/moviecritc/test.imdb.php in the browser, I get the following error:

http://i.imgur.com/YGRxG.png

Can you please help identify this? Thanks.

Is there anyway cast can be

Submitted by Alex (not verified) on

Is there anyway cast can be grabbed?. I am looking at this script and but I need a way to grab the cast. Is it possible?

Yes, I have added the ability

Submitted by alif on

Yes, I have added the ability to grab cast now :)

Nice script... Its simple and

Submitted by Andy (not verified) on

Nice script...
Its simple and easy to use. Thanks for sharing.

Hello Alif, Thank you. I was

Submitted by EJee (not verified) on

Hello Alif,

Thank you. I was looking for information on how to do this myself when I stumbled on your code. This is directly usable. Nice work.

I have one question. I would also like to grab the "User Rating:" but I have trouble locating the text.

Is this possible to do at all?

Thanks in advance,
EJEE

Yes, its definitely possible.

Submitted by alif on

Yes, its definitely possible. Infact, I just updated my class to reflect it. Now, the class should return User Rating and Total Votes by default. But, you can turn it off by calling the Method showRating(false).

Thanks,

any chance you can get

Submitted by Anonymous (not verified) on

any chance you can get celebrity details eg from http://www.imdb.com/name/nm1200692/??

why dont this wor $movieInfo

Submitted by Anonymous (not verified) on

why dont this wor

$movieInfo = $imdbObj
->add('Archive Footage:', '/div/a')
->get('http://www.imdb.com/name/nm0004710/');

http://www.imdb.com/name/nm1200692/

Hello, It won't work for

Submitted by alif on

Hello, It won't work for Grabbing Celebrity details from a Celebrity page. This Grabber works for Grabbing Movie Details of a Movie. For example, it will work for:

http://www.imdb.com/title/tt0320661/

Or even if you type 'Kingdom of Heaven', that should also work (for the above result).

Hey Alif, thanks for this

Submitted by Dennis (not verified) on

Hey Alif,

thanks for this nice script (-:
but how can i show the results in my Page with HTML?

like "echo "Film: " + $movieInfo[????];

I m not the biggest Profi (-: Sorry..

wou8ld nice if you help me (-:

Thanks, Dennis

Hi Dennis, If you want to

Submitted by alif on

Hi Dennis,

If you want to display the infos in php pages, then you can use:

Film: =$movieInfo['title:']?>
Plot: =$movieInfo['Plot:']?>
Director: =$movieInfo['Director:']?>

...etc. and so on (if you use the above syntax, make sure short_tags are turned on in your php settings).

You can view all the associated array keys in the test.imdb.php class.

I hope it helps,

This is an great piece of

Submitted by Kevin (not verified) on

This is an great piece of script. Its much easier to use.

Fatal error (this could be

Submitted by Anonymous (not verified) on

Fatal error (this could be when the IMDB query fails)

PHP Fatal error: Call to a member function getAttribute() on a non-object

public function getImdbURL($url) {
$searchURL = 'http://www.imdb.com/find?q='.urlencode($url);
$searchDom = new DomDocument();
$searchLoad = $searchDom->loadHTMLFile($searchURL);
$xpath = new DomXPath($searchDom);
return 'http://www.imdb.com'.$xpath->query('//p[@style]/b/a')->item(0)->getAttribute('href');
}

Thanks for the catch. I

Submitted by alif on

Thanks for the catch.

I noticed it throws an error when an invalid search term (say: 'x98359835') is entered. In which case, Imdb doesn't return any valid Movie.

The error also occurs if you enter an invalid url (say 'http://www.google.ca/'), which isn't a valid imdb url.

Both the issues should be fixed now. Please download the updated files (class.imdb.php.txt and test.imdb.php.txt).

Thanks,

Well done - a failed search

Submitted by Anonymous (not verified) on

Well done - a failed search (eg, "Shutter Island") now returns no results and not a FATAL error reported by PHP.

The search term is valid, it's just that no results are obtained.

Thanks

I just noticed the error. It

Submitted by alif on

I just noticed the error. It seems for some search results, Imdb simply redirects to the Movie URL without showing any search Result. For others, Imdb returns a listing of Movies as strong match.

For example: If you search 'The Matrix'.
http://www.imdb.com/find?s=all&q=Matrix

Imdb will return a search page with the Search Result.

Whereas, if you type 'Shutter Island', if simply redirects to the movie page without displaying the results.
http://www.imdb.com/find?s=all&q=Shutter+Island

Redirects to: http://www.imdb.com/title/tt1130884/

without showing any search Result. Thanks for pointing the issue. I have fixed it for now.

If there are any other issues you have come across, let me know,

Thanks,

When I use your IMDB demo

Submitted by Anonymous (not verified) on

When I use your IMDB demo with a title + year (eg. "The Italian Job (1969)"), I get a correct result. When I use your script locally, nothing is returned.

Any ideas ?

Thats strange...I just tried

Submitted by alif on

Thats strange...I just tried it locally and it seems to work for me. Can you please explain what error you are getting?. Are you getting no result (just blank) or something else.

Thanks,

$IMDB emtpy when $Title is

Submitted by Anonymous (not verified) on

$IMDB emtpy when $Title is "The Italian Job (1969)", but
is filled when "The Italian Job"

$ErrorLevel = error_reporting(E_ERROR);
$imdbObj = new Imdb();
$IMDB = $imdbObj->showCast(true)
->add('Genre:', '/div/a')
->add('Tagline:', '/div/text()')
->add('Certification:', '/div/a')
->get($Title);
error_reporting($ErrorLevel);

echo "IMDB(".$Title.")";
echo '';
print_r($IMDB);
echo '';

I tried the code and it seems

Submitted by alif on

I tried the code and it seems to be working for me. So, I am not sure why you are getting that error :| :S

This is what I tried (same as yours):

$Title = 'The Italian Job (1969)';
$IMDB = $imdbObj->showCast(true)
->add('Genre:', '/div/a')
->add('Tagline:', '/div/text()')
->add('Certification:', '/div/a')
->get($Title);

echo "

";
print_r($IMDB);
echo "
";

The above works for me. Have you tried other search queries, do they work out for you?or do you get empty return as well?

The code on my blog (Demo Page) and the code for download is same.

It was last modified on July 8th, 2010 for a minor fix.

HI, i'm on the road of

Submitted by Wisdom (not verified) on

HI,
i'm on the road of learning web scraping
can't plz exmplai how to keep formatting while scrapping content because my scraped article are not formatted so useless

I a trying very hard to make

Submitted by Anonymous (not verified) on

I a trying very hard to make a copy of the example page. When you type in a movie title or IMDB url i just need it to return the info when I hit grab.

I am able to get it to show the examples but Ive had no luck figuring out how to make it search and bring back the results.

Any help would be greatly appreciated. Thank you.

I have tested your script on

Submitted by J (not verified) on

I have tested your script on two separate servers, both running PHP 5.2.14 with all aspects of DOM enabled, on one it loads everything, on the other Shutter Island doesn't load, instead it gives a Warning about DOMDocument being empty after using loadHTML and a Fatal Exception when attempting to use getAttribute on a non DOMDocument object. Do you know what could be causing the problem? Thanks.

Hi, Sorry, I am not sure why

Submitted by alif on

Hi,

Sorry, I am not sure why its not working out for you.

For me, it seems to work fine.. :S..

Hey, This is just what I was

Submitted by Dan (not verified) on

Hey,

This is just what I was looking for, however I am getting alot of error messages when I try to use this class to get information from IMDB.

Ex:
Warning: DOMDocument::loadHTMLFile() [domdocument.loadhtmlfile]: Tag layer invalid in http://www.imdb.com/find?q=Taken, line: 113 in /Applications/XAMPP/xamppfiles/htdocs/film_system/etc/imdb.class.php on line 241

Warning: DOMDocument::loadHTMLFile() [domdocument.loadhtmlfile]: htmlParseEntityRef: no name in http://www.imdb.com/find?q=Taken, line: 191 in /Applications/XAMPP/xamppfiles/htdocs/film_system/etc/imdb.class.php on line 241

They come from the two lines in your class.imdb.php document that contain loadHTMLFile(), ex:

public function getImdbURL($url) {
$queryStr = '//p[@style]/b/a';
$validTitleStr = "//head/link[@rel='canonical']";
$searchURL = 'http://www.imdb.com/find?q='.urlencode($url);
$searchDom = new DomDocument();
*********$searchLoad = $searchDom->loadHTMLFile($searchURL);*********

$xpath = new DomXPath($searchDom);
// check to see if Imdb has directly redirected to the movie page. It happens for some movies (Ex: Shutter Island as someone pointed out).
$linkHrefURL = $xpath->query($validTitleStr)->item(0)->getAttribute('href');
if($this->isValidURL($linkHrefURL) ) {
return $linkHrefURL;
}

if($xpath->query($queryStr)->length > 0 ) {
return 'http://www.imdb.com'.$xpath->query($queryStr)->item(0)->getAttribute('href');
} else {
return false;
}
}
}

Any ideas on what I am doing wrong?

I found out that the script

Submitted by Dan (not verified) on

I found out that the script works, it get the information, but for some reason I get all of those DOMDocument errors.

A ideas on how to get them to go away?

Best regards,
Dan

PS: Awsome script :)

Hi Dan, Thanks for trying out

Submitted by alif on

Hi Dan,

Thanks for trying out the script. These are actually warnings not errors in PHP. They happen because the IMDB HTML page are not well formed and/or are not Valid HTML.

At the top of your page where you are calling the class.imdb.php, you can use the following line:

// only report errors in PHP and ignore warnings.
error_reporting(E_ERROR);

I have also used this line on the test.imdb.php file provided on the blog.

I hope this helps,

Your script is really nice

Submitted by Jay (not verified) on

Your script is really nice but my script need to show search result from imdb and then grab infos... how can i do that? Thank you very much. :)

hello...i im trying to use

Submitted by ronovar (not verified) on

hello...i im trying to use this excellent script but can't to make to work to fetch following data:

studio, writer, producer, cast, audiencerating and rating.

for example i use folloving code:

$movieInfo = $imdbObj->add('Writer:', '/div/text()');

echo $movieInfo['Writer:']."";

but result is empty.

here is imdb source:

Writer:

Barry Levy

but i think that problem is /name/nm1633356/

so could fix this?

many thanks.

This script not working to

Submitted by apu (not verified) on

This script not working to new IMDB layout. do you have updated script?

Thank you for pointing it

Submitted by alif on

Thank you for pointing it out. I just noticed IMDB changed their layout completely. I will look into it asap.

Thanks again.

I updated my script to grab

Submitted by alif on

I updated my script to grab from the new Layout. It seems to be working fine with no issues.

Thanks,

Great script! Thank you. I

Submitted by Alex (not verified) on

Great script! Thank you.

I would just add few lines to capture the "Release date": Starting from line 160 I repeated the code in the loop above with a small modification:

$nodeNameList=$xpath->query('h4',$nodeList->item($i));
$grabName=trim($nodeNameList->item(0)->nodeValue);
$nodeValueList=$xpath->query('text()', $nodeList->item($i));
$grabValue[$grabName]=$this->getValue('/text()',$nodeValueList);

Alex

Hey, Thanks for trying out. I

Submitted by alif on

Hey, Thanks for trying out. I have added the Release Date now, all you need to do is add an extra line on the constructor (around line 86) to get the Release Date.

$this->defaultList = array (
.....
'Release Date:' => '/text()'
);

This should solve it.

Thanks,

thanks, i hope this works

Submitted by ruwan (not verified) on

thanks, i hope this works

http://www.imdb.com/title/tt0

Submitted by sameer (not verified) on

http://www.imdb.com/title/tt0860462/

this IMDB link dosen't work. how to fix it
thank you

HEy, can you publish your php

Submitted by maxur (not verified) on

HEy,
can you publish your php from your demo page, its very good formated and without the array,.........

Big Thx

Gret

I love this script. Is there

Submitted by ADT (not verified) on

I love this script. Is there anyway to get the list of genres for the movie?

Hey there, my suggestion is

Submitted by Alex (not verified) on

Hey there,

my suggestion is to add the following lines after grabbing the storyline

// grab genres:
$grabValue['Genres:'] = "";
$genresPath = $xpath->query("//div[@class='see-more inline canwrap'][h4='Genres:']/a/text()");
for($i=0;$i < $genresPath->length; $i++) {
$grabValue['Genres:'] .= $genresPath->item($i)->nodeValue .", ";
}
$grabValue['Genres:'] = rtrim($grabValue['Genres:']," ,");

Again, great script!
Alex

Thanks.. There's still some

Submitted by alif on

Thanks.. There's still some polishing that needs to be done on the class. I have been too busy and have not been able to sit with it.

Thanks again for the genre addition =).

Thanks! That works perfectly.

Submitted by ADt (not verified) on

Thanks! That works perfectly.

Pages

Add new comment