Showing posts with label javascript. Show all posts
Showing posts with label javascript. Show all posts

Saturday, February 19, 2011

Headless HTML page rendering with phantomjs

What is phantomjs?


phantomjs is a headless browser which can render HTML pages into images. It uses a Webkit rendering engine. That's cool because the generated page images aren't missing any dynamically loaded javascript stuff. Event flash movies are shown as expected.



Building phantomjs


The phantomjs homepage contains some very useful hints about getting and building phantomjs. Read the comments at the end of the build instructions page if something doesn't work out for you. There are many useful hints.



Rendering my first page


phantomjs is controlled using javascript commands. You can launch your phantomjs javascript files from the commandline:



$ phantomjs myScript.js



phantomjs calls your script every time after a page is loaded. To react in different ways on different pages you have to track your current "state". You can persist your script's state in the var phantom.state. phantomjs restores the value of phantom.state even after another page is loaded. Other javascript variables disappear after loading a new page. Initially the state is empty.



To load a page call phantom.open(url). phantomjs will load the page below the specified URL and execute your script after the page's DOM is loaded.



The (in my opinion) coolest function is phantom.render(path). phantomjs will save a "screenshot" of your current page to the specified path on your hard drive. The path's file extension defines the image file format. Your viewport's size can be defined by setting phantom.viewportSize.



After you're done with your phantom things you have to call phantom.exit(returnCode) to quit. The returnCode is passed as status to the parent process.



Now putting all that functions together in a script gives us the following:



if(phantom.state.length === 0){
phantom.state = '0_home';
phantom.open('http://www.mini.de');
}
else if(phantom.state === '0_home'){
phantom.viewportSize = {width: 800, height: 600};
phantom.sleep(2000);
phantom.render('home.png');
phantom.exit(0);
}



The result after executing the script is a screenshot of www.mini.de in a file named 'home.png'. The image will show the main stage with a flash movie and a footer with three dynamically loaded HTML fragments. You can see the result below:





Clicking links


"Clicking links" like a humanoid user is simulated by firing mouse click events. The following listing defines the clickElement(id) function which can be used to "click" elements. This allows us to execute simple page flows.



function clickElement(id){
var a = document.getElementById(id);

var e = document.createEvent('MouseEvents');
e.initMouseEvent('click', true, true, window, 0, 0, 0, 0, 0, false, false, false, false, 0, null);

a.dispatchEvent(e);
}

if(phantom.state.length !== 0){
// save screenshot for every page / state
phantom.viewportSize = {width: 800, height: 600};
phantom.sleep(2000);
phantom.render('screen_' + phantom.state + '.png');
}


if(phantom.state.length === 0){
phantom.state = '0_home';
phantom.open('http://www.mini.de');
}
else if(phantom.state === '0_home'){
phantom.state = '1_config';

clickElement('quicklink_id1');
}
else if(phantom.state === '1_config'){
phantom.exit();
}



The script will load the URL http://www.mini.de which is the home page. After loading the home page the element with the ID 'quicklink_id1' will be target of a click event. The element with the ID 'quicklink_id1' should be the 'MINI KONFIGURATOR' link in the footer.



Asserting stuff


Within phantomjs scripts you can access you page's DOM, the global javascript variables and global javascript functions. This enables us to do some kind of unit testing.



I'm extending the listing once more:



function clickElement(id){ ... }

function fail(msg){
console.log(msg);

phantom.exit(1);
}

function assert(condition, msg){
if(condition){
return;
}

fail(msg);
}


if(phantom.state.length !== 0){
phantom.viewportSize = {width: 800, height: 600};
phantom.sleep(2000);
phantom.render('screen_' + phantom.state + '.png');
}

if(phantom.state.length === 0){
phantom.state = '0_home';
phantom.open('http://www.mini.de');
}
else if(phantom.state === '0_home'){
phantom.state = '1_config';

clickElement('quicklink_id1');
}
else if(phantom.state === '1_config'){
assert(document.getElementById('cake'), 'I am missing the cake!');

phantom.exit();
}



Check this out: phantomjs returns a status code of 1 when the assertion fails. This can be used to do some custom post processing in the shell:



$ phantomjs myScript.js && echo 'Test successful' || echo 'Test failed'



Further information


For further information see the following pages. They are sorted by what I think is important.


Monday, February 26, 2007

Update of Doom

So among the things I was up to today, updating the website was one of them. I had previously thrown in a ton of AJAX and javascript for the site but never actually put them to use, instead simply laying the foundations for a later date. While I am not completely finished with this AJAX thing, I did manage to accomplish quite a bit for this round. And then something entirely odd happened...

Firefox decided it no longer wished to display PNG files. Actually, I should rephrase that - Firefox refused to display one particular PNG file I was trying to use for the new design. The image in question happened to be the new header image with the AW Gate, of which I originally saved as banner1.png on the server. No big deal, I figured, since I had already used PNG format for most of the site.

So I go to load it up in Firefox to check what it looks like and it simply doesn't show up.

Weird... so I crack open Internet Explorer 7 and it shows up fine. Stranger still. Then I ask a friend to check out the page using Opera (you know, the code nazi) and it shows up in Opera. Now I'm totally baffled...

Once again I open Firefox and still it doesn't show up. At this point I start tearing apart the HTML by hand to see if there is a missing div tag or something that IE7 would overlook... no dice. For all intents and purposes, the code was fine and it should have worked.

So after about an hour of tearing out my hair and using words reserved for less of a family occasion, I finally just open Photoshop thinking "This is a stupid reason for Firefox not to see this... it can't possibly be the reason..."

So I open the graphic and simply rename it. Uploaded it to the server and bingo... it worked in firefox.

What the heck just happened here? The world may never know... in any event, along with using the reflect tag for some images on the site (I had previously set the framework up for this but never used it) I also redid the Updates section to match the theme of the website. Again, I used a framework I had set up but previously didn't use (being the AJAX RSS feeder) to pull the RSS for this section you are reading into the custom layout for the website.

While I was at it, I decided to make a few more graphics for the website (logo for the Metaverse EX listing on the products page, and also the AW2EX) and throw the reflect on them. So all in all, things look good. There are still some things I need to work out for later (like the header for the templates section) and I am thinking about changing the Updates link to read News instead - apparently this may be confusing for some readers.

I also decided (out of shear boredom) to place a Digg link on the first page as a private joke to try and get a story about VR5 Online buried. Seriously, I wanted people to bury the story just out of boredom... lol. Either way I didn't care, but just wanted to see what would happen.

Also changed was the wording for the About Us section to reflect our laid back attitude (or more importantly, mine) and also to announce that VR5 Online apparently qualifies as a Web 3.0 company. Seriously I have no idea what that means, I was trying to validate some AJAX on the site to make sure it registered and the site that did the validation said we do not qualify as a Web 2.0 company, to which I raised an eyebrow.

Further down the list it was overjoyed to inform me that we were a Web 3.0 company instead. Now I'm just outright confused, but my train of thought says that 3.0 is better than 2.0 any day. So if Web 2.0 is the next big thing, then Web 3.0 must be like a religious experience to geeks or something...

Anyway, that's my news for today... other than that, I'm still messing around with the AJAX stuff. Oh wait.. one more thing...

I noticed recently that there seems to be two prominent frameworks for AJAX being used. In one corner you have Prototype and in another JQuery. Now, I know people from either camp could go on for years about the benefits of either, but here's the deal - I use JQuery simply because it more or less makes my life that much easier to code. I haven't had a chance to mess with Prototype in detail yet, but it seems a hell of a lot harder to work with than JQuery.

So what is the concern? Well the Web 2.0 validator looks specifically for Prototype when checking for Web 2.0 and not JQuery (or both). So many of the things that would normally qualify as Web 2.0 and AJAX simply are a no show to this validator because there is no instance of Prototype being used.

Just something that came to mind on the spur of the moment. Why is it that Web 2.0 has to be ridiculously hard to deal with in order to garner any credit? I mean, if somebody creates an easy to use framework, then why try to discredit it?

I don't think I'll even understand how this works... and trust me, I have over ten years in sociology, so you would think I would be able to have a bit of insight. After all of these years, the only thing I know for sure is that people in general are lunatics.

And I mean that in the nicest possible manner, of course...

Heavily Medicated and Enjoying the Padded Room - Darian Knight