Grep the web with Fresno: a command line for Firefox
Ben from Simile points me to Fresno, a tool that connects to a MozRepl-equipped Firefox and drives it from the command line.
Fresno can make a running Firefox navigate to URLs, load JavaScript files, and execute JavaScript commands. It keeps the browser as the execution context or changes it to the currently loaded web page or arbitrary objects. This example from the documentation retrieves links from a web page:
% ./fresno -p http://simile.mit.edu/ -c \
-j "document.getElementById('slideshow').innerHTML" \
| grep href
<div class="title"><a href="semantic-bank/">Semantic Bank</a></div>
<div class="title"><a href="gadget/">Gadget</a></div>
<div class="title"><a href="welkin/">Welkin</a></div>
<div class="title"><a href="timeline/">Timeline</a></div>
<div class="title"><a href="referee/">Referee</a></div>
<div class="title"><a href="babel/">Babel</a></div>
<div class="title"><a href="exhibit/">Exhibit</a></div>
<div class="title"><a href="appalachian/">Appalachian</a></div>
I’m pleased to report that, despite being surrounded by nothing else than ink-black X terminals, the little red panda is starting to feel very much at home on my Unix desktop.
Update 2007-07-04: ZIGOROu also points me to his MozRepl Perl module!
I had made Perl module about MozRepl.
http://search.cpan.org/dist/MozRepl/
MozRepl module is pluggable. Example,
http://search.cpan.org/dist/MozRepl-Plugin-Restart/
http://search.cpan.org/dist/MozRepl-Plugin-LinkTools/
Sweet. I’ve updated the post and linked both on MozRepl page, later I’ll add a wiki page for each. Thanks!
I'm looking for any scripts or javascripts or special ones to capture one special target of content in a web then copy it into my local database... dya have any tips or solution to help me ? tx
Any chance to have a MozRepl in python? That would be great...
M.E.
A MozRepl in Python would b great indeed - any news on that? Sounds like a great way to automate load tests.
Post new comment