Reading remote file contents with PHP+CURL

The preferred method to read contents of a file into a string with PHP is usually file_get_contents(), but sometimes server configurations do not allow for file_get_contents() to read from a remote server. Luckily CURL provides a nice workaround (you must have the libcurl package installed on your version of PHP). Here's how I retrieve data from my anime list on AniDB.net for use on anime.zakness.com:

The function: get_contents()

  1. // returns a String containing the contents of a URL
  2. function get_contents($url, $post="") {
  3. $ch = curl_init();
  4. curl_setopt($ch, CURLOPT_URL, $url);
  5. curl_setopt($ch, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);
  6. if (!empty($post)) {
  7. curl_setopt($ch, CURLOPT_POST,1);
  8. curl_setopt($ch, CURLOPT_POSTFIELDS, $post);
  9. }
  10. curl_setopt($ch, CURLOPT_HEADER, 0);
  11. curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
  12. curl_setopt($ch, CURLOPT_ENCODING, "");
  13. $string = curl_exec($ch) or die(curl_error());
  14. curl_close($ch);
  15. return $string;
  16. }
  17. source file: get-contents.txt

Using get_contents() to access anidb.net

I can use get_contents() to access the source code of my AniDB MyList (which needs to send POST information to AniDB's server). Then I use preg_match_all() to get all the relevant information and populate the data array ($data).

  1. $data = array();
  2. $errors = array();
  3. $source = get_contents("http://anidb.info/perl-bin/animedb.pl", "show=mylist&xuser=zaclin&xpass=******");
  4. if (empty($source)) array_push($errors, "Could not retrieve raw source data.");
  5. preg_match_all("@class=\"title\".+aid=(\d+)\">(.+)<\/a>.+(\d+?)\s*\/\s*(\d+?).+(\d+?)\s*\/\s*\d+@Usx", $source, $m, PREG_SET_ORDER);
  6. if (!$m) array_push($errors,"Could not find matching data.");
  7. else {
  8. foreach ($m as $match) {
  9. $d = array();
  10. $d['id'] = $match[1];
  11. $d['name'] = $match[2];
  12. $d['have'] = $match[3];
  13. $d['total'] = $match[4];
  14. $d['seen'] = $match[5];
  15. $data[] = $d;
  16. }
  17. }
  18. source file: get-contents-example.txt

These two snippets do all the dirty work for me, now I can do whatever I want with the data. Not too shabby!