How relevant are these posts now?

http://blog.ianbicking.org/2008/12/10/lxml-an-underappreciat...

http://blog.ianbicking.org/2008/03/30/python-html-parser-per...

Very.

I recently switched from BeautifulSoup to lxml. It's tons faster, and I like it quite a bit more than BeautifulSoup. The documentation isn't quite as good, and it's a bit of a pain to install, but if you have to do a bunch of scraping, I'd check it out.

What pain? I've only used lxml for a project on Windows, but there, it was as simple as installing setuptools and then 'easy-install lxml'.

I think the 'pain' comes when there are issues with the c library that lxml binds to.

Python has Scrapy (http://scrapy.org), quoting: "Scrapy is a fast high-level screen scraping and web crawling framework, used to crawl websites and extract structured data from their pages. It can be used for a wide range of purposes, from data mining to monitoring and automated testing."

I've been working on a Java HTML parser: http://jsoup.org/

It supports CSS selectors, jquery-like DOM traversal and set manipulation, etc.

I was under the impression BeautifulSoup is no longer actively maintained.

(Edit: Which makes me sad, because I loved their documentation writing style)

It's not :(

No idea.